2023-10-18T15:44:31.510 INFO:root:teuthology version: 0.0.1.dev201+g202b180 2023-10-18T15:44:31.511 DEBUG:teuthology.run:Teuthology command: teuthology --description rbd/cli_v1/{base/install clusters/{fixed-1 openstack} features/format-1 msgr-failures/few objectstore/bluestore-stupid pool/small-cache-pool supported-random-distro$/{rhel_8} workloads/rbd_cli_generic} --archive /home/teuthworker/archive/yuriw-2023-10-17_20:38:29-rbd-wip-yuri2-testing-2023-10-11-0853-pacific-distro-default-smithi/7431146 --verbose --owner scheduled_yuriw@teuthology --name yuriw-2023-10-17_20:38:29-rbd-wip-yuri2-testing-2023-10-11-0853-pacific-distro-default-smithi -- /home/teuthworker/archive/yuriw-2023-10-17_20:38:29-rbd-wip-yuri2-testing-2023-10-11-0853-pacific-distro-default-smithi/7431146/orig.config.yaml 2023-10-18T15:44:31.545 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-18T15:44:31.620 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-17_20:38:29-rbd-wip-yuri2-testing-2023-10-11-0853-pacific-distro-default-smithi/7431146 branch: wip-yuri2-testing-2023-10-11-0853-pacific description: rbd/cli_v1/{base/install clusters/{fixed-1 openstack} features/format-1 msgr-failures/few objectstore/bluestore-stupid pool/small-cache-pool supported-random-distro$/{rhel_8} workloads/rbd_cli_generic} email: null first_in_suite: false job_id: '7431146' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2023-10-17_20:38:29-rbd-wip-yuri2-testing-2023-10-11-0853-pacific-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 3 size: 30 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri2-testing-2023-10-11-0853-pacific ceph: conf: client: rbd default format: 1 global: mon client directed command retry: 5 ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: stupid bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(CACHE_POOL_NEAR_FULL\) - \(CACHE_POOL_NO_HIT_SET\) sha1: 84698e743199450ee5f6969d22ebe6bed3a68e4a ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log global: osd crush chooseleaf type: 0 osd pool default pg num: 128 osd pool default pgp num: 128 osd pool default size: 2 mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 84698e743199450ee5f6969d22ebe6bed3a68e4a selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri2-testing-2023-10-11-0853-pacific sha1: 84698e743199450ee5f6969d22ebe6bed3a68e4a owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 2214 sha1: 84698e743199450ee5f6969d22ebe6bed3a68e4a sleep_before_teardown: 0 subset: 111/120000 suite: rbd suite_branch: wip-yuri2-testing-2023-10-11-0853-pacific suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_84698e743199450ee5f6969d22ebe6bed3a68e4a/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 84698e743199450ee5f6969d22ebe6bed3a68e4a targets: smithi082.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOivSv8XGJ5NGv6c6zD4SdEvKnmlMEGAwjHGFVunXQn0rlpezRBKpVUINTonVtsg+ed/sV3IayPLsYHwtPK5Rqw= tasks: - install: null - ceph: null - exec: client.0: - sudo ceph osd pool create cache 4 - sudo ceph osd tier add rbd cache - sudo ceph osd tier cache-mode cache writeback - sudo ceph osd tier set-overlay rbd cache - sudo ceph osd pool set cache hit_set_type bloom - sudo ceph osd pool set cache hit_set_count 8 - sudo ceph osd pool set cache hit_set_period 60 - sudo ceph osd pool set cache target_max_objects 250 - workunit: clients: client.0: - rbd/cli_generic.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 202b180cb047e798fb131047314a862593f45403 timestamp: 2023-10-17_20:38:29 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-18T15:44:31.620 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_84698e743199450ee5f6969d22ebe6bed3a68e4a/qa; will attempt to use it 2023-10-18T15:44:31.621 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_84698e743199450ee5f6969d22ebe6bed3a68e4a/qa/tasks 2023-10-18T15:44:31.621 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-18T15:44:31.623 INFO:teuthology.task.internal:Checking packages... 2023-10-18T15:44:31.647 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '84698e743199450ee5f6969d22ebe6bed3a68e4a' 2023-10-18T15:44:31.647 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-18T15:44:31.647 INFO:teuthology.packaging:ref: None 2023-10-18T15:44:31.648 INFO:teuthology.packaging:tag: None 2023-10-18T15:44:31.648 INFO:teuthology.packaging:branch: wip-yuri2-testing-2023-10-11-0853-pacific 2023-10-18T15:44:31.648 INFO:teuthology.packaging:sha1: 84698e743199450ee5f6969d22ebe6bed3a68e4a 2023-10-18T15:44:31.649 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri2-testing-2023-10-11-0853-pacific 2023-10-18T15:44:31.913 INFO:teuthology.task.internal:Found packages for ceph version 16.2.14-75.g84698e74 2023-10-18T15:44:31.914 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-18T15:44:31.921 INFO:teuthology.task.internal:no buildpackages task found 2023-10-18T15:44:31.921 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-18T15:44:31.927 INFO:teuthology.task.internal:Saving configuration 2023-10-18T15:44:31.940 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-18T15:44:31.969 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-18T15:44:31.995 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-17_20:38:29-rbd-wip-yuri2-testing-2023-10-11-0853-pacific-distro-default-smithi/7431146', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 15:39:35.206702', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOivSv8XGJ5NGv6c6zD4SdEvKnmlMEGAwjHGFVunXQn0rlpezRBKpVUINTonVtsg+ed/sV3IayPLsYHwtPK5Rqw='} 2023-10-18T15:44:31.995 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-18T15:44:32.000 INFO:teuthology.task.internal:roles: ubuntu@smithi082.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2023-10-18T15:44:32.001 INFO:teuthology.run_tasks:Running task console_log... 2023-10-18T15:44:32.044 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe45fda8a60>, signals=[15]) 2023-10-18T15:44:32.045 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-18T15:44:32.051 INFO:teuthology.task.internal:Opening connections... 2023-10-18T15:44:32.052 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi082.front.sepia.ceph.com 2023-10-18T15:44:32.053 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T15:44:32.119 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-18T15:44:32.126 DEBUG:teuthology.orchestra.run.smithi082:> uname -m 2023-10-18T15:44:32.188 INFO:teuthology.orchestra.run.smithi082.stdout:x86_64 2023-10-18T15:44:32.189 DEBUG:teuthology.orchestra.run.smithi082:> cat /etc/os-release 2023-10-18T15:44:32.243 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="Red Hat Enterprise Linux" 2023-10-18T15:44:32.243 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="8.4 (Ootpa)" 2023-10-18T15:44:32.243 INFO:teuthology.orchestra.run.smithi082.stdout:ID="rhel" 2023-10-18T15:44:32.244 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE="fedora" 2023-10-18T15:44:32.244 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="8.4" 2023-10-18T15:44:32.244 INFO:teuthology.orchestra.run.smithi082.stdout:PLATFORM_ID="platform:el8" 2023-10-18T15:44:32.244 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-18T15:44:32.244 INFO:teuthology.orchestra.run.smithi082.stdout:ANSI_COLOR="0;31" 2023-10-18T15:44:32.245 INFO:teuthology.orchestra.run.smithi082.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-18T15:44:32.245 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://www.redhat.com/" 2023-10-18T15:44:32.245 INFO:teuthology.orchestra.run.smithi082.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-18T15:44:32.245 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-18T15:44:32.246 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:44:32.246 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-18T15:44:32.246 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-18T15:44:32.246 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-18T15:44:32.246 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-18T15:44:32.247 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2023-10-18T15:44:32.273 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-18T15:44:32.283 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-18T15:44:32.296 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-18T15:44:32.296 DEBUG:teuthology.orchestra.run.smithi082:> test '!' -e /home/ubuntu/cephtest 2023-10-18T15:44:32.310 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-18T15:44:32.317 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-18T15:44:32.317 DEBUG:teuthology.orchestra.run.smithi082:> test -z $(ls -A /var/lib/ceph) 2023-10-18T15:44:32.365 INFO:teuthology.orchestra.run.smithi082.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-18T15:44:32.366 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-18T15:44:32.402 INFO:teuthology.run_tasks:Running task kernel... 2023-10-18T15:44:32.416 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-10-18T15:44:32.417 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-10-18T15:44:32.417 DEBUG:teuthology.orchestra.run.smithi082:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-18T15:44:32.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T15:44:32.432 DEBUG:teuthology.orchestra.run.smithi082:> uname -r 2023-10-18T15:44:32.486 INFO:teuthology.orchestra.run.smithi082.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-18T15:44:32.486 INFO:teuthology.task.kernel:Running kernel on smithi082: 4.18.0-305.25.1.el8_4.x86_64 2023-10-18T15:44:32.487 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum install -y kernel 2023-10-18T15:44:34.114 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:44:34.115 INFO:teuthology.orchestra.run.smithi082.stdout:Unable to read consumer identity 2023-10-18T15:44:35.583 INFO:teuthology.orchestra.run.smithi082.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2023-10-18T15:44:36.109 INFO:teuthology.orchestra.run.smithi082.stdout:Extra Packages for Enterprise Linux 30 MB/s | 16 MB 00:00 2023-10-18T15:44:39.428 INFO:teuthology.orchestra.run.smithi082.stdout:lab-extras 3.9 MB/s | 24 kB 00:00 2023-10-18T15:44:40.231 INFO:teuthology.orchestra.run.smithi082.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-10-18T15:44:40.232 INFO:teuthology.orchestra.run.smithi082.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-10-18T15:44:40.232 INFO:teuthology.orchestra.run.smithi082.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-10-18T15:44:40.389 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-10-18T15:44:40.390 INFO:teuthology.orchestra.run.smithi082.stdout:Nothing to do. 2023-10-18T15:44:40.390 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-10-18T15:44:40.493 DEBUG:teuthology.orchestra.run.smithi082:> echo no | sudo yum reinstall kernel || true 2023-10-18T15:44:40.836 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:44:40.837 INFO:teuthology.orchestra.run.smithi082.stdout:Unable to read consumer identity 2023-10-18T15:44:41.123 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:00:02 ago on Wed 18 Oct 2023 03:44:39 PM UTC. 2023-10-18T15:44:41.249 INFO:teuthology.orchestra.run.smithi082.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-18T15:44:41.250 INFO:teuthology.orchestra.run.smithi082.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-18T15:44:41.250 INFO:teuthology.orchestra.run.smithi082.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-18T15:44:41.283 INFO:teuthology.orchestra.run.smithi082.stderr:Error: No packages marked for reinstall. 2023-10-18T15:44:41.332 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum reinstall -y kernel || true 2023-10-18T15:44:41.674 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:44:41.674 INFO:teuthology.orchestra.run.smithi082.stdout:Unable to read consumer identity 2023-10-18T15:44:41.957 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:00:02 ago on Wed 18 Oct 2023 03:44:39 PM UTC. 2023-10-18T15:44:42.102 INFO:teuthology.orchestra.run.smithi082.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-18T15:44:42.103 INFO:teuthology.orchestra.run.smithi082.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-18T15:44:42.103 INFO:teuthology.orchestra.run.smithi082.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-18T15:44:42.124 INFO:teuthology.orchestra.run.smithi082.stderr:Error: No packages marked for reinstall. 2023-10-18T15:44:42.173 DEBUG:teuthology.orchestra.run.smithi082:> rpm -q kernel | sort -rV | head -n 1 2023-10-18T15:44:42.298 INFO:teuthology.orchestra.run.smithi082.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-18T15:44:42.298 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-18T15:44:42.299 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-10-18T15:44:42.299 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-18T15:44:42.299 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-18T15:44:42.299 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-10-18T15:44:42.300 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi082.front.sepia.ceph.com, path=None, version=distro) 2023-10-18T15:44:42.300 DEBUG:teuthology.orchestra.run.smithi082:> rpm -q kernel | sort -rV | head -n 1 2023-10-18T15:44:42.424 INFO:teuthology.orchestra.run.smithi082.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-18T15:44:42.425 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-18T15:44:42.425 DEBUG:teuthology.orchestra.run.smithi082:> sudo rpm -qi grub2-tools 2023-10-18T15:44:42.452 INFO:teuthology.orchestra.run.smithi082.stdout:Name : grub2-tools 2023-10-18T15:44:42.452 INFO:teuthology.orchestra.run.smithi082.stdout:Epoch : 1 2023-10-18T15:44:42.452 INFO:teuthology.orchestra.run.smithi082.stdout:Version : 2.02 2023-10-18T15:44:42.453 INFO:teuthology.orchestra.run.smithi082.stdout:Release : 99.el8_4.1 2023-10-18T15:44:42.453 INFO:teuthology.orchestra.run.smithi082.stdout:Architecture: x86_64 2023-10-18T15:44:42.453 INFO:teuthology.orchestra.run.smithi082.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-18T15:44:42.453 INFO:teuthology.orchestra.run.smithi082.stdout:Group : System Environment/Base 2023-10-18T15:44:42.453 INFO:teuthology.orchestra.run.smithi082.stdout:Size : 9264631 2023-10-18T15:44:42.453 INFO:teuthology.orchestra.run.smithi082.stdout:License : GPLv3+ 2023-10-18T15:44:42.454 INFO:teuthology.orchestra.run.smithi082.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-18T15:44:42.454 INFO:teuthology.orchestra.run.smithi082.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-18T15:44:42.454 INFO:teuthology.orchestra.run.smithi082.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-18T15:44:42.454 INFO:teuthology.orchestra.run.smithi082.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-18T15:44:42.455 INFO:teuthology.orchestra.run.smithi082.stdout:Relocations : (not relocatable) 2023-10-18T15:44:42.455 INFO:teuthology.orchestra.run.smithi082.stdout:Packager : Red Hat, Inc. 2023-10-18T15:44:42.455 INFO:teuthology.orchestra.run.smithi082.stdout:Vendor : Red Hat, Inc. 2023-10-18T15:44:42.455 INFO:teuthology.orchestra.run.smithi082.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-18T15:44:42.455 INFO:teuthology.orchestra.run.smithi082.stdout:Summary : Support tools for GRUB. 2023-10-18T15:44:42.456 INFO:teuthology.orchestra.run.smithi082.stdout:Description : 2023-10-18T15:44:42.456 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:44:42.456 INFO:teuthology.orchestra.run.smithi082.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-18T15:44:42.456 INFO:teuthology.orchestra.run.smithi082.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-18T15:44:42.456 INFO:teuthology.orchestra.run.smithi082.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-18T15:44:42.456 INFO:teuthology.orchestra.run.smithi082.stdout:hardware devices. 2023-10-18T15:44:42.457 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:44:42.457 INFO:teuthology.orchestra.run.smithi082.stdout:This subpackage provides tools for support of all platforms. 2023-10-18T15:44:42.458 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-18T15:44:42.458 INFO:teuthology.task.kernel:Updating grub on smithi082 to boot 4.18.0-425.3.1.el8.x86_64 2023-10-18T15:44:42.458 DEBUG:teuthology.orchestra.run.smithi082:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-18T15:44:43.914 INFO:teuthology.orchestra.run.smithi082.stderr:Generating grub configuration file ... 2023-10-18T15:44:45.937 INFO:teuthology.orchestra.run.smithi082.stderr:done 2023-10-18T15:44:45.938 DEBUG:teuthology.orchestra.run.smithi082:> mktemp 2023-10-18T15:44:45.953 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/tmp.Eo1AeVkiMj 2023-10-18T15:44:45.953 DEBUG:teuthology.orchestra.run.smithi082:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Eo1AeVkiMj 2023-10-18T15:44:46.026 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0666 /tmp/tmp.Eo1AeVkiMj 2023-10-18T15:44:46.180 DEBUG:teuthology.orchestra.remote:smithi082:/tmp/tmp.Eo1AeVkiMj is 6KB 2023-10-18T15:44:46.228 DEBUG:teuthology.orchestra.run.smithi082:> rm -fr /tmp/tmp.Eo1AeVkiMj 2023-10-18T15:44:46.242 DEBUG:teuthology.orchestra.run.smithi082:> sudo /bin/ls /boot/loader/entries || true 2023-10-18T15:44:46.325 INFO:teuthology.orchestra.run.smithi082.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-18T15:44:46.325 INFO:teuthology.orchestra.run.smithi082.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-18T15:44:46.325 INFO:teuthology.orchestra.run.smithi082.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-18T15:44:46.326 INFO:teuthology.orchestra.run.smithi082.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-18T15:44:46.327 DEBUG:teuthology.orchestra.run.smithi082:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-10-18T15:44:46.469 DEBUG:teuthology.orchestra.run.smithi082:> sudo shutdown -r now 2023-10-18T15:45:16.484 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-10-18T15:45:16.485 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2023-10-18T15:45:16.486 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T15:45:34.846 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.82 2023-10-18T15:45:43.856 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2023-10-18T15:45:43.857 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T15:45:46.938 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.82 2023-10-18T15:45:58.950 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2023-10-18T15:45:58.951 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T15:46:31.450 DEBUG:teuthology.orchestra.run.smithi082:> true 2023-10-18T15:46:32.852 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi082.front.sepia.ceph.com' 2023-10-18T15:46:32.852 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-425.3.1.el8.x86_64"... 2023-10-18T15:46:32.853 DEBUG:teuthology.orchestra.run.smithi082:> uname -r 2023-10-18T15:46:32.961 INFO:teuthology.orchestra.run.smithi082.stdout:4.18.0-425.3.1.el8.x86_64 2023-10-18T15:46:32.962 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-10-18T15:46:32.962 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-18T15:46:32.962 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-18T15:46:33.963 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-10-18T15:46:33.964 DEBUG:teuthology.orchestra.run.smithi082:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-18T15:46:34.283 INFO:teuthology.orchestra.run.smithi082.stdout:ttyS1 2023-10-18T15:46:34.296 DEBUG:teuthology.parallel:result is None 2023-10-18T15:46:34.297 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-18T15:46:34.304 INFO:teuthology.task.internal:Creating test directory... 2023-10-18T15:46:34.305 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-18T15:46:34.321 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-18T15:46:34.329 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-18T15:46:34.339 INFO:teuthology.task.internal:Creating archive directory... 2023-10-18T15:46:34.340 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-18T15:46:34.388 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-18T15:46:34.395 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-18T15:46:34.395 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-18T15:46:34.458 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T15:46:34.469 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T15:46:34.470 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-18T15:46:34.478 INFO:teuthology.task.internal:Configuring sudo... 2023-10-18T15:46:34.478 DEBUG:teuthology.orchestra.run.smithi082:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-18T15:46:34.536 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-18T15:46:34.544 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-18T15:46:34.545 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-18T15:46:34.591 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-18T15:46:34.674 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-18T15:46:34.714 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-18T15:46:34.796 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-18T15:46:34.835 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-10-18T15:46:34.835 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-18T15:46:34.899 DEBUG:teuthology.orchestra.run.smithi082:> sudo service rsyslog restart 2023-10-18T15:46:34.992 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-18T15:46:35.344 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-18T15:46:35.352 INFO:teuthology.task.internal:Starting timer... 2023-10-18T15:46:35.352 INFO:teuthology.run_tasks:Running task pcp... 2023-10-18T15:46:35.373 INFO:teuthology.run_tasks:Running task selinux... 2023-10-18T15:46:35.388 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-10-18T15:46:35.389 DEBUG:teuthology.orchestra.run.smithi082:> sudo service auditd rotate 2023-10-18T15:46:35.444 INFO:teuthology.orchestra.run.smithi082.stdout:Rotating logs: [ OK ] 2023-10-18T15:46:35.446 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-18T15:46:35.446 DEBUG:teuthology.orchestra.run.smithi082:> /usr/sbin/getenforce 2023-10-18T15:46:35.474 INFO:teuthology.orchestra.run.smithi082.stdout:Permissive 2023-10-18T15:46:35.474 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi082.front.sepia.ceph.com': 'permissive'} 2023-10-18T15:46:35.475 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-18T15:46:35.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T15:46:35.540 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-18T15:46:35.541 DEBUG:teuthology.orchestra.run.smithi082:> sudo /usr/sbin/setenforce permissive 2023-10-18T15:46:35.605 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-18T15:46:35.616 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-18T15:46:35.617 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-18T15:46:35.634 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-18T15:46:35.636 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi082.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-18T15:54:49.480 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi082.front.sepia.ceph.com')] 2023-10-18T15:54:49.482 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2023-10-18T15:54:49.483 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T15:54:49.552 DEBUG:teuthology.orchestra.run.smithi082:> true 2023-10-18T15:54:49.620 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi082.front.sepia.ceph.com' 2023-10-18T15:54:49.621 INFO:teuthology.run_tasks:Running task clock... 2023-10-18T15:54:49.632 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-18T15:54:49.632 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-18T15:54:49.633 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T15:54:49.689 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-18T15:54:49.703 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-18T15:54:49.736 INFO:teuthology.orchestra.run.smithi082.stderr:sudo: ntpd: command not found 2023-10-18T15:54:49.746 INFO:teuthology.orchestra.run.smithi082.stdout:506 Cannot talk to daemon 2023-10-18T15:54:49.760 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-18T15:54:49.774 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-18T15:54:49.842 INFO:teuthology.orchestra.run.smithi082.stderr:bash: ntpq: command not found 2023-10-18T15:54:49.845 INFO:teuthology.orchestra.run.smithi082.stdout:210 Number of sources = 4 2023-10-18T15:54:49.845 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-18T15:54:49.845 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2023-10-18T15:54:49.846 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T15:54:49.846 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T15:54:49.846 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T15:54:49.846 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T15:54:49.847 INFO:teuthology.run_tasks:Running task install... 2023-10-18T15:54:49.856 DEBUG:teuthology.task.install:project ceph 2023-10-18T15:54:49.857 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '84698e743199450ee5f6969d22ebe6bed3a68e4a'}} 2023-10-18T15:54:49.857 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '84698e743199450ee5f6969d22ebe6bed3a68e4a'} 2023-10-18T15:54:49.857 INFO:teuthology.task.install:Using flavor: default 2023-10-18T15:54:49.863 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-18T15:54:49.864 INFO:teuthology.task.install:extra packages: [] 2023-10-18T15:54:49.864 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '84698e743199450ee5f6969d22ebe6bed3a68e4a', 'tag': None, 'wait_for_package': False} 2023-10-18T15:54:49.865 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=84698e743199450ee5f6969d22ebe6bed3a68e4a 2023-10-18T15:54:50.015 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-11-0853-pacific/84698e743199450ee5f6969d22ebe6bed3a68e4a/centos/8/flavors/default/ 2023-10-18T15:54:50.016 INFO:teuthology.task.install.rpm:Package version is 16.2.14-75.g84698e74 2023-10-18T15:54:50.126 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-11-0853-pacific/84698e743199450ee5f6969d22ebe6bed3a68e4a/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-11-0853-pacific/84698e743199450ee5f6969d22ebe6bed3a68e4a/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-11-0853-pacific/84698e743199450ee5f6969d22ebe6bed3a68e4a/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-18T15:54:50.126 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-10-18T15:54:50.127 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-18T15:54:50.152 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-10-18T15:54:50.153 DEBUG:teuthology.orchestra.run.smithi082:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/84698e743199450ee5f6969d22ebe6bed3a68e4a/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-18T15:54:50.218 DEBUG:teuthology.orchestra.run.smithi082:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-18T15:54:50.294 DEBUG:teuthology.orchestra.run.smithi082:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-18T15:54:50.361 INFO:teuthology.orchestra.run.smithi082.stdout:check_obsoletes = 1 2023-10-18T15:54:50.361 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum clean all 2023-10-18T15:54:50.750 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:54:51.058 INFO:teuthology.orchestra.run.smithi082.stdout:44 files removed 2023-10-18T15:54:51.109 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install ceph-radosgw 2023-10-18T15:54:51.457 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:54:52.686 INFO:teuthology.orchestra.run.smithi082.stdout:ceph packages for x86_64 95 kB/s | 76 kB 00:00 2023-10-18T15:54:53.214 INFO:teuthology.orchestra.run.smithi082.stdout:ceph noarch packages 32 kB/s | 15 kB 00:00 2023-10-18T15:54:53.759 INFO:teuthology.orchestra.run.smithi082.stdout:ceph source packages 2.8 kB/s | 1.4 kB 00:00 2023-10-18T15:54:53.894 INFO:teuthology.orchestra.run.smithi082.stdout:Copr repo for python3-asyncssh owned by ceph 33 kB/s | 3.5 kB 00:00 2023-10-18T15:54:56.239 INFO:teuthology.orchestra.run.smithi082.stdout:Extra Packages for Enterprise Linux 6.9 MB/s | 16 MB 00:02 2023-10-18T15:54:59.569 INFO:teuthology.orchestra.run.smithi082.stdout:lab-extras 4.1 MB/s | 24 kB 00:00 2023-10-18T15:54:59.925 INFO:teuthology.orchestra.run.smithi082.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 120 MB/s | 33 MB 00:00 2023-10-18T15:55:05.047 INFO:teuthology.orchestra.run.smithi082.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 128 MB/s | 38 MB 00:00 2023-10-18T15:55:13.831 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-10-18T15:55:13.834 INFO:teuthology.orchestra.run.smithi082.stdout:=================================================================================================== 2023-10-18T15:55:13.834 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2023-10-18T15:55:13.834 INFO:teuthology.orchestra.run.smithi082.stdout:=================================================================================================== 2023-10-18T15:55:13.835 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2023-10-18T15:55:13.835 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw x86_64 2:16.2.14-75.g84698e74.el8 ceph 12 M 2023-10-18T15:55:13.835 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrading: 2023-10-18T15:55:13.835 INFO:teuthology.orchestra.run.smithi082.stdout: librados2 x86_64 2:16.2.14-75.g84698e74.el8 ceph 3.7 M 2023-10-18T15:55:13.835 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1 x86_64 2:16.2.14-75.g84698e74.el8 ceph 4.2 M 2023-10-18T15:55:13.836 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2023-10-18T15:55:13.836 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base x86_64 2:16.2.14-75.g84698e74.el8 ceph 6.1 M 2023-10-18T15:55:13.836 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common x86_64 2:16.2.14-75.g84698e74.el8 ceph 24 M 2023-10-18T15:55:13.836 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux x86_64 2:16.2.14-75.g84698e74.el8 ceph 26 k 2023-10-18T15:55:13.837 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-18T15:55:13.837 INFO:teuthology.orchestra.run.smithi082.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2023-10-18T15:55:13.837 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2 x86_64 2:16.2.14-75.g84698e74.el8 ceph 824 k 2023-10-18T15:55:13.837 INFO:teuthology.orchestra.run.smithi082.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-18T15:55:13.837 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-18T15:55:13.838 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1 x86_64 2:16.2.14-75.g84698e74.el8 ceph 493 k 2023-10-18T15:55:13.838 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-18T15:55:13.838 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2 x86_64 2:16.2.14-75.g84698e74.el8 ceph 3.8 M 2023-10-18T15:55:13.838 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-18T15:55:13.838 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-18T15:55:13.839 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse x86_64 2:16.2.14-75.g84698e74.el8 ceph 45 k 2023-10-18T15:55:13.839 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common x86_64 2:16.2.14-75.g84698e74.el8 ceph 99 k 2023-10-18T15:55:13.839 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs x86_64 2:16.2.14-75.g84698e74.el8 ceph 215 k 2023-10-18T15:55:13.839 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-18T15:55:13.839 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados x86_64 2:16.2.14-75.g84698e74.el8 ceph 390 k 2023-10-18T15:55:13.840 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd x86_64 2:16.2.14-75.g84698e74.el8 ceph 369 k 2023-10-18T15:55:13.840 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw x86_64 2:16.2.14-75.g84698e74.el8 ceph 114 k 2023-10-18T15:55:13.840 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:55:13.840 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2023-10-18T15:55:13.840 INFO:teuthology.orchestra.run.smithi082.stdout:=================================================================================================== 2023-10-18T15:55:13.841 INFO:teuthology.orchestra.run.smithi082.stdout:Install 21 Packages 2023-10-18T15:55:13.841 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrade 2 Packages 2023-10-18T15:55:13.841 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:55:13.842 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 57 M 2023-10-18T15:55:13.842 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2023-10-18T15:55:17.138 INFO:teuthology.orchestra.run.smithi082.stdout:(1/23): ceph-radosgw-16.2.14-75.g84698e74.el8.x 3.7 MB/s | 12 MB 00:03 2023-10-18T15:55:17.180 INFO:teuthology.orchestra.run.smithi082.stdout:(2/23): ceph-selinux-16.2.14-75.g84698e74.el8.x 608 kB/s | 26 kB 00:00 2023-10-18T15:55:17.280 INFO:teuthology.orchestra.run.smithi082.stdout:(3/23): libcephfs2-16.2.14-75.g84698e74.el8.x86 8.1 MB/s | 824 kB 00:00 2023-10-18T15:55:17.340 INFO:teuthology.orchestra.run.smithi082.stdout:(4/23): libradosstriper1-16.2.14-75.g84698e74.e 8.0 MB/s | 493 kB 00:00 2023-10-18T15:55:17.372 INFO:teuthology.orchestra.run.smithi082.stdout:(5/23): ceph-base-16.2.14-75.g84698e74.el8.x86_ 1.8 MB/s | 6.1 MB 00:03 2023-10-18T15:55:17.512 INFO:teuthology.orchestra.run.smithi082.stdout:(6/23): python3-ceph-argparse-16.2.14-75.g84698 322 kB/s | 45 kB 00:00 2023-10-18T15:55:17.552 INFO:teuthology.orchestra.run.smithi082.stdout:(7/23): python3-ceph-common-16.2.14-75.g84698e7 2.4 MB/s | 99 kB 00:00 2023-10-18T15:55:17.606 INFO:teuthology.orchestra.run.smithi082.stdout:(8/23): librgw2-16.2.14-75.g84698e74.el8.x86_64 14 MB/s | 3.8 MB 00:00 2023-10-18T15:55:17.643 INFO:teuthology.orchestra.run.smithi082.stdout:(9/23): python3-cephfs-16.2.14-75.g84698e74.el8 2.3 MB/s | 215 kB 00:00 2023-10-18T15:55:17.661 INFO:teuthology.orchestra.run.smithi082.stdout:(10/23): python3-rados-16.2.14-75.g84698e74.el8 6.9 MB/s | 390 kB 00:00 2023-10-18T15:55:17.714 INFO:teuthology.orchestra.run.smithi082.stdout:(11/23): python3-rgw-16.2.14-75.g84698e74.el8.x 2.2 MB/s | 114 kB 00:00 2023-10-18T15:55:17.777 INFO:teuthology.orchestra.run.smithi082.stdout:(12/23): python3-rbd-16.2.14-75.g84698e74.el8.x 2.7 MB/s | 369 kB 00:00 2023-10-18T15:55:18.355 INFO:teuthology.orchestra.run.smithi082.stdout:(13/23): ceph-common-16.2.14-75.g84698e74.el8.x 5.6 MB/s | 24 MB 00:04 2023-10-18T15:55:19.260 INFO:teuthology.orchestra.run.smithi082.stdout:(14/23): gperftools-libs-2.7-9.el8.x86_64.rpm 198 kB/s | 306 kB 00:01 2023-10-18T15:55:19.371 INFO:teuthology.orchestra.run.smithi082.stdout:(15/23): liboath-2.6.2-3.el8.x86_64.rpm 59 kB/s | 59 kB 00:00 2023-10-18T15:55:19.413 INFO:teuthology.orchestra.run.smithi082.stdout:(16/23): python3-prettytable-0.7.2-14.el8.noarc 1.0 MB/s | 44 kB 00:00 2023-10-18T15:55:19.439 INFO:teuthology.orchestra.run.smithi082.stdout:(17/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 9.7 MB/s | 259 kB 00:00 2023-10-18T15:55:19.467 INFO:teuthology.orchestra.run.smithi082.stdout:(18/23): librdkafka-0.11.4-1.el8.x86_64.rpm 13 MB/s | 353 kB 00:00 2023-10-18T15:55:19.483 INFO:teuthology.orchestra.run.smithi082.stdout:(19/23): leveldb-1.22-1.el8.x86_64.rpm 106 kB/s | 181 kB 00:01 2023-10-18T15:55:19.492 INFO:teuthology.orchestra.run.smithi082.stdout:(20/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.9 MB/s | 47 kB 00:00 2023-10-18T15:55:19.502 INFO:teuthology.orchestra.run.smithi082.stdout:(21/23): libunwind-1.3.1-3.el8.x86_64.rpm 310 kB/s | 75 kB 00:00 2023-10-18T15:55:19.939 INFO:teuthology.orchestra.run.smithi082.stdout:(22/23): librbd1-16.2.14-75.g84698e74.el8.x86_6 9.4 MB/s | 4.2 MB 00:00 2023-10-18T15:55:20.573 INFO:teuthology.orchestra.run.smithi082.stdout:(23/23): librados2-16.2.14-75.g84698e74.el8.x86 3.4 MB/s | 3.7 MB 00:01 2023-10-18T15:55:20.580 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2023-10-18T15:55:20.580 INFO:teuthology.orchestra.run.smithi082.stdout:Total 8.5 MB/s | 57 MB 00:06 2023-10-18T15:55:20.585 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2023-10-18T15:55:20.633 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2023-10-18T15:55:20.634 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2023-10-18T15:55:21.360 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2023-10-18T15:55:21.365 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2023-10-18T15:55:22.400 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2023-10-18T15:55:22.622 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 1/25 2023-10-18T15:55:22.993 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 1/25 2023-10-18T15:55:23.214 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librados2-2:16.2.14-75.g84698e74.el8.x86_64 2/25 2023-10-18T15:55:23.319 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-2:16.2.14-75.g84698e74.el8.x86_64 2/25 2023-10-18T15:55:23.504 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rados-2:16.2.14-75.g84698e74.el8.x86_64 3/25 2023-10-18T15:55:23.787 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/25 2023-10-18T15:55:23.989 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs2-2:16.2.14-75.g84698e74.el8.x86_64 5/25 2023-10-18T15:55:24.399 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephfs2-2:16.2.14-75.g84698e74.el8.x86_64 5/25 2023-10-18T15:55:24.622 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librbd1-2:16.2.14-75.g84698e74.el8.x86_64 6/25 2023-10-18T15:55:24.697 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-2:16.2.14-75.g84698e74.el8.x86_64 6/25 2023-10-18T15:55:24.904 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 7/25 2023-10-18T15:55:25.097 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 8/25 2023-10-18T15:55:25.478 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 8/25 2023-10-18T15:55:25.655 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librgw2-2:16.2.14-75.g84698e74.el8.x86_64 9/25 2023-10-18T15:55:25.721 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librgw2-2:16.2.14-75.g84698e74.el8.x86_64 9/25 2023-10-18T15:55:25.878 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : leveldb-1.22-1.el8.x86_64 10/25 2023-10-18T15:55:26.028 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-argparse-2:16.2.14-75.g84698e74.el8.x 11/25 2023-10-18T15:55:26.218 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cephfs-2:16.2.14-75.g84698e74.el8.x86_64 12/25 2023-10-18T15:55:26.448 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rgw-2:16.2.14-75.g84698e74.el8.x86_64 13/25 2023-10-18T15:55:26.671 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rbd-2:16.2.14-75.g84698e74.el8.x86_64 14/25 2023-10-18T15:55:26.846 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libradosstriper1-2:16.2.14-75.g84698e74.el8.x86_64 15/25 2023-10-18T15:55:26.916 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libradosstriper1-2:16.2.14-75.g84698e74.el8.x86_64 15/25 2023-10-18T15:55:27.090 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-10-18T15:55:27.281 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 17/25 2023-10-18T15:55:27.563 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-10-18T15:55:27.755 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-common-2:16.2.14-75.g84698e74.el8.x86 19/25 2023-10-18T15:55:32.573 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:16.2.14-75.g84698e74.el8.x86_64 20/25 2023-10-18T15:55:32.796 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-common-2:16.2.14-75.g84698e74.el8.x86_64 20/25 2023-10-18T15:55:33.512 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:16.2.14-75.g84698e74.el8.x86_64 20/25 2023-10-18T15:55:33.809 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-base-2:16.2.14-75.g84698e74.el8.x86_64 21/25 2023-10-18T15:55:33.886 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-base-2:16.2.14-75.g84698e74.el8.x86_64 21/25 2023-10-18T15:55:33.995 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-selinux-2:16.2.14-75.g84698e74.el8.x86_64 22/25 2023-10-18T15:55:51.256 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-selinux-2:16.2.14-75.g84698e74.el8.x86_64 22/25 2023-10-18T15:55:51.257 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /sys 2023-10-18T15:55:51.257 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /proc 2023-10-18T15:55:51.257 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /dev 2023-10-18T15:55:51.257 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /run 2023-10-18T15:55:51.258 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /mnt 2023-10-18T15:55:51.258 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /var/tmp 2023-10-18T15:55:51.258 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /home 2023-10-18T15:55:51.258 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /tmp 2023-10-18T15:55:51.259 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /dev 2023-10-18T15:55:51.259 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:55:52.471 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-radosgw-2:16.2.14-75.g84698e74.el8.x86_64 23/25 2023-10-18T15:55:52.556 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-75.g84698e74.el8.x86_64 23/25 2023-10-18T15:55:52.556 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-10-18T15:55:52.776 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-10-18T15:55:52.777 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-10-18T15:55:53.575 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-10-18T15:55:53.575 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-base-2:16.2.14-75.g84698e74.el8.x86_64 1/25 2023-10-18T15:55:53.575 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-common-2:16.2.14-75.g84698e74.el8.x86_64 2/25 2023-10-18T15:55:53.576 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-radosgw-2:16.2.14-75.g84698e74.el8.x86_64 3/25 2023-10-18T15:55:53.576 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-selinux-2:16.2.14-75.g84698e74.el8.x86_64 4/25 2023-10-18T15:55:53.576 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs2-2:16.2.14-75.g84698e74.el8.x86_64 5/25 2023-10-18T15:55:53.576 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libradosstriper1-2:16.2.14-75.g84698e74.el8.x86_64 6/25 2023-10-18T15:55:53.577 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librgw2-2:16.2.14-75.g84698e74.el8.x86_64 7/25 2023-10-18T15:55:53.577 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-argparse-2:16.2.14-75.g84698e74.el8.x 8/25 2023-10-18T15:55:53.577 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-common-2:16.2.14-75.g84698e74.el8.x86 9/25 2023-10-18T15:55:53.577 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cephfs-2:16.2.14-75.g84698e74.el8.x86_64 10/25 2023-10-18T15:55:53.577 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rados-2:16.2.14-75.g84698e74.el8.x86_64 11/25 2023-10-18T15:55:53.578 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rbd-2:16.2.14-75.g84698e74.el8.x86_64 12/25 2023-10-18T15:55:53.578 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rgw-2:16.2.14-75.g84698e74.el8.x86_64 13/25 2023-10-18T15:55:53.578 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 14/25 2023-10-18T15:55:53.578 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : leveldb-1.22-1.el8.x86_64 15/25 2023-10-18T15:55:53.578 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 16/25 2023-10-18T15:55:53.579 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 17/25 2023-10-18T15:55:53.579 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-10-18T15:55:53.579 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 19/25 2023-10-18T15:55:53.579 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 20/25 2023-10-18T15:55:53.579 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 21/25 2023-10-18T15:55:53.580 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-2:16.2.14-75.g84698e74.el8.x86_64 22/25 2023-10-18T15:55:53.580 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-10-18T15:55:53.580 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-2:16.2.14-75.g84698e74.el8.x86_64 24/25 2023-10-18T15:55:54.954 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-10-18T15:55:54.954 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:00:50 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:55:57.042 INFO:teuthology.orchestra.run.smithi082.stdout:Installed products updated. 2023-10-18T15:55:57.146 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:55:57.146 INFO:teuthology.orchestra.run.smithi082.stdout:Upgraded: 2023-10-18T15:55:57.146 INFO:teuthology.orchestra.run.smithi082.stdout: librados2-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:55:57.147 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:55:57.147 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2023-10-18T15:55:57.147 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:55:57.147 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:55:57.147 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:55:57.148 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:55:57.148 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-18T15:55:57.148 INFO:teuthology.orchestra.run.smithi082.stdout: leveldb-1.22-1.el8.x86_64 2023-10-18T15:55:57.148 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:55:57.149 INFO:teuthology.orchestra.run.smithi082.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-18T15:55:57.149 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-18T15:55:57.149 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:55:57.149 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-18T15:55:57.149 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:55:57.150 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-18T15:55:57.150 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-18T15:55:57.150 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:55:57.150 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:55:57.150 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:55:57.151 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-18T15:55:57.151 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:55:57.151 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:55:57.151 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:55:57.151 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:55:57.152 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-10-18T15:55:57.355 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install ceph-test 2023-10-18T15:55:57.719 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:55:58.673 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:00:54 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:56:00.484 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-10-18T15:56:00.486 INFO:teuthology.orchestra.run.smithi082.stdout:=========================================================================================== 2023-10-18T15:56:00.486 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2023-10-18T15:56:00.486 INFO:teuthology.orchestra.run.smithi082.stdout:=========================================================================================== 2023-10-18T15:56:00.487 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2023-10-18T15:56:00.487 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test x86_64 2:16.2.14-75.g84698e74.el8 ceph 52 M 2023-10-18T15:56:00.487 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2023-10-18T15:56:00.487 INFO:teuthology.orchestra.run.smithi082.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-18T15:56:00.487 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite x86_64 2:16.2.14-75.g84698e74.el8 ceph 174 k 2023-10-18T15:56:00.488 INFO:teuthology.orchestra.run.smithi082.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-18T15:56:00.488 INFO:teuthology.orchestra.run.smithi082.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-18T15:56:00.488 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-18T15:56:00.488 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:56:00.488 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2023-10-18T15:56:00.488 INFO:teuthology.orchestra.run.smithi082.stdout:=========================================================================================== 2023-10-18T15:56:00.489 INFO:teuthology.orchestra.run.smithi082.stdout:Install 6 Packages 2023-10-18T15:56:00.489 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:56:00.489 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 53 M 2023-10-18T15:56:00.490 INFO:teuthology.orchestra.run.smithi082.stdout:Installed size: 196 M 2023-10-18T15:56:00.490 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2023-10-18T15:56:00.491 INFO:teuthology.orchestra.run.smithi082.stdout:(1/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 18 MB/s | 70 kB 00:00 2023-10-18T15:56:00.549 INFO:teuthology.orchestra.run.smithi082.stdout:(2/6): jq-1.5-12.el8.x86_64.rpm 2.7 MB/s | 161 kB 00:00 2023-10-18T15:56:00.584 INFO:teuthology.orchestra.run.smithi082.stdout:(3/6): oniguruma-6.8.2-2.el8.x86_64.rpm 5.4 MB/s | 187 kB 00:00 2023-10-18T15:56:00.617 INFO:teuthology.orchestra.run.smithi082.stdout:(4/6): socat-1.7.3.3-2.el8.x86_64.rpm 8.9 MB/s | 302 kB 00:00 2023-10-18T15:56:00.836 INFO:teuthology.orchestra.run.smithi082.stdout:(5/6): libcephsqlite-16.2.14-75.g84698e74.el8.x 499 kB/s | 174 kB 00:00 2023-10-18T15:56:04.355 INFO:teuthology.orchestra.run.smithi082.stdout:(6/6): ceph-test-16.2.14-75.g84698e74.el8.x86_6 13 MB/s | 52 MB 00:03 2023-10-18T15:56:04.358 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2023-10-18T15:56:04.359 INFO:teuthology.orchestra.run.smithi082.stdout:Total 14 MB/s | 53 MB 00:03 2023-10-18T15:56:04.362 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2023-10-18T15:56:04.385 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2023-10-18T15:56:04.385 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2023-10-18T15:56:04.713 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2023-10-18T15:56:04.717 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2023-10-18T15:56:05.226 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2023-10-18T15:56:05.424 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 1/6 2023-10-18T15:56:05.638 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 2/6 2023-10-18T15:56:05.728 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 2/6 2023-10-18T15:56:05.914 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : jq-1.5-12.el8.x86_64 3/6 2023-10-18T15:56:06.112 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 4/6 2023-10-18T15:56:06.238 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephsqlite-2:16.2.14-75.g84698e74.el8.x86_64 5/6 2023-10-18T15:56:10.885 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephsqlite-2:16.2.14-75.g84698e74.el8.x86_64 5/6 2023-10-18T15:56:11.118 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-test-2:16.2.14-75.g84698e74.el8.x86_64 6/6 2023-10-18T15:56:11.387 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-test-2:16.2.14-75.g84698e74.el8.x86_64 6/6 2023-10-18T15:56:11.387 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-test-2:16.2.14-75.g84698e74.el8.x86_64 1/6 2023-10-18T15:56:11.387 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephsqlite-2:16.2.14-75.g84698e74.el8.x86_64 2/6 2023-10-18T15:56:11.387 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 3/6 2023-10-18T15:56:11.388 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : jq-1.5-12.el8.x86_64 4/6 2023-10-18T15:56:11.388 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 5/6 2023-10-18T15:56:12.506 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 6/6 2023-10-18T15:56:12.506 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:01:08 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:56:14.429 INFO:teuthology.orchestra.run.smithi082.stdout:Installed products updated. 2023-10-18T15:56:14.531 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:56:14.531 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2023-10-18T15:56:14.532 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:56:14.532 INFO:teuthology.orchestra.run.smithi082.stdout: jq-1.5-12.el8.x86_64 2023-10-18T15:56:14.532 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:56:14.532 INFO:teuthology.orchestra.run.smithi082.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-18T15:56:14.532 INFO:teuthology.orchestra.run.smithi082.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-18T15:56:14.533 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-18T15:56:14.533 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:56:14.533 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-10-18T15:56:14.655 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install ceph 2023-10-18T15:56:15.008 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:56:15.934 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:01:11 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:56:17.734 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-10-18T15:56:17.737 INFO:teuthology.orchestra.run.smithi082.stdout:====================================================================================================== 2023-10-18T15:56:17.737 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2023-10-18T15:56:17.738 INFO:teuthology.orchestra.run.smithi082.stdout:====================================================================================================== 2023-10-18T15:56:17.738 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2023-10-18T15:56:17.738 INFO:teuthology.orchestra.run.smithi082.stdout: ceph x86_64 2:16.2.14-75.g84698e74.el8 ceph 6.4 k 2023-10-18T15:56:17.738 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2023-10-18T15:56:17.739 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds x86_64 2:16.2.14-75.g84698e74.el8 ceph 2.4 M 2023-10-18T15:56:17.739 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr x86_64 2:16.2.14-75.g84698e74.el8 ceph 1.6 M 2023-10-18T15:56:17.739 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core noarch 2:16.2.14-75.g84698e74.el8 ceph-noarch 244 k 2023-10-18T15:56:17.739 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon x86_64 2:16.2.14-75.g84698e74.el8 ceph 5.0 M 2023-10-18T15:56:17.739 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd x86_64 2:16.2.14-75.g84698e74.el8 ceph 18 M 2023-10-18T15:56:17.740 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-18T15:56:17.740 INFO:teuthology.orchestra.run.smithi082.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-18T15:56:17.740 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-18T15:56:17.740 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-18T15:56:17.741 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-18T15:56:17.741 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-18T15:56:17.741 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-18T15:56:17.741 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-18T15:56:17.741 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-18T15:56:17.742 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-18T15:56:17.742 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-18T15:56:17.742 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-18T15:56:17.742 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-18T15:56:17.742 INFO:teuthology.orchestra.run.smithi082.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-18T15:56:17.743 INFO:teuthology.orchestra.run.smithi082.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-18T15:56:17.743 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-18T15:56:17.743 INFO:teuthology.orchestra.run.smithi082.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-18T15:56:17.743 INFO:teuthology.orchestra.run.smithi082.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-18T15:56:17.744 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-18T15:56:17.744 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-18T15:56:17.744 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-18T15:56:17.744 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-18T15:56:17.744 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:56:17.745 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2023-10-18T15:56:17.745 INFO:teuthology.orchestra.run.smithi082.stdout:====================================================================================================== 2023-10-18T15:56:17.745 INFO:teuthology.orchestra.run.smithi082.stdout:Install 28 Packages 2023-10-18T15:56:17.745 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:56:17.746 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 30 M 2023-10-18T15:56:17.746 INFO:teuthology.orchestra.run.smithi082.stdout:Installed size: 96 M 2023-10-18T15:56:17.746 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2023-10-18T15:56:18.315 INFO:teuthology.orchestra.run.smithi082.stdout:(1/28): ceph-16.2.14-75.g84698e74.el8.x86_64.rp 30 kB/s | 6.4 kB 00:00 2023-10-18T15:56:18.502 INFO:teuthology.orchestra.run.smithi082.stdout:(2/28): ceph-mds-16.2.14-75.g84698e74.el8.x86_6 6.1 MB/s | 2.4 MB 00:00 2023-10-18T15:56:18.507 INFO:teuthology.orchestra.run.smithi082.stdout:(3/28): ceph-mgr-16.2.14-75.g84698e74.el8.x86_6 3.9 MB/s | 1.6 MB 00:00 2023-10-18T15:56:19.137 INFO:teuthology.orchestra.run.smithi082.stdout:(4/28): ceph-mgr-modules-core-16.2.14-75.g84698 388 kB/s | 244 kB 00:00 2023-10-18T15:56:19.474 INFO:teuthology.orchestra.run.smithi082.stdout:(5/28): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 130 kB/s | 44 kB 00:00 2023-10-18T15:56:19.725 INFO:teuthology.orchestra.run.smithi082.stdout:(6/28): python3-beautifulsoup4-4.6.3-2.el8.1.no 736 kB/s | 185 kB 00:00 2023-10-18T15:56:19.910 INFO:teuthology.orchestra.run.smithi082.stdout:(7/28): ceph-osd-16.2.14-75.g84698e74.el8.x86_6 13 MB/s | 18 MB 00:01 2023-10-18T15:56:19.912 INFO:teuthology.orchestra.run.smithi082.stdout:(8/28): python3-cheroot-8.5.2-1.el8.noarch.rpm 926 kB/s | 173 kB 00:00 2023-10-18T15:56:19.996 INFO:teuthology.orchestra.run.smithi082.stdout:(9/28): python3-jaraco-6.2-6.el8.noarch.rpm 132 kB/s | 11 kB 00:00 2023-10-18T15:56:20.018 INFO:teuthology.orchestra.run.smithi082.stdout:(10/28): ceph-mon-16.2.14-75.g84698e74.el8.x86_ 3.0 MB/s | 5.0 MB 00:01 2023-10-18T15:56:20.080 INFO:teuthology.orchestra.run.smithi082.stdout:(11/28): python3-jaraco-functools-2.0-4.el8.noa 210 kB/s | 18 kB 00:00 2023-10-18T15:56:20.164 INFO:teuthology.orchestra.run.smithi082.stdout:(12/28): python3-more-itertools-7.2.0-3.el8.noa 696 kB/s | 59 kB 00:00 2023-10-18T15:56:20.333 INFO:teuthology.orchestra.run.smithi082.stdout:(13/28): python3-pecan-1.3.2-9.el8.noarch.rpm 1.6 MB/s | 283 kB 00:00 2023-10-18T15:56:20.353 INFO:teuthology.orchestra.run.smithi082.stdout:(14/28): python3-logutils-0.3.5-11.el8.noarch.r 145 kB/s | 49 kB 00:00 2023-10-18T15:56:20.416 INFO:teuthology.orchestra.run.smithi082.stdout:(15/28): python3-cherrypy-18.4.0-1.el8.noarch.r 761 kB/s | 384 kB 00:00 2023-10-18T15:56:20.417 INFO:teuthology.orchestra.run.smithi082.stdout:(16/28): python3-portend-2.6-1.el8.noarch.rpm 185 kB/s | 16 kB 00:00 2023-10-18T15:56:20.437 INFO:teuthology.orchestra.run.smithi082.stdout:(17/28): python3-simplegeneric-0.8.1-17.el8.noa 222 kB/s | 19 kB 00:00 2023-10-18T15:56:20.500 INFO:teuthology.orchestra.run.smithi082.stdout:(18/28): python3-singledispatch-3.4.0.3-18.el8. 285 kB/s | 24 kB 00:00 2023-10-18T15:56:20.501 INFO:teuthology.orchestra.run.smithi082.stdout:(19/28): python3-tempora-1.14.1-5.el8.noarch.rp 347 kB/s | 29 kB 00:00 2023-10-18T15:56:20.522 INFO:teuthology.orchestra.run.smithi082.stdout:(20/28): python3-trustme-0.6.0-4.el8.noarch.rpm 325 kB/s | 27 kB 00:00 2023-10-18T15:56:20.586 INFO:teuthology.orchestra.run.smithi082.stdout:(21/28): python3-waitress-1.4.3-1.el8.noarch.rp 2.9 MB/s | 255 kB 00:00 2023-10-18T15:56:20.588 INFO:teuthology.orchestra.run.smithi082.stdout:(22/28): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.8 MB/s | 251 kB 00:00 2023-10-18T15:56:20.650 INFO:teuthology.orchestra.run.smithi082.stdout:(23/28): python3-werkzeug-0.12.2-4.el8.noarch.r 7.3 MB/s | 457 kB 00:00 2023-10-18T15:56:20.671 INFO:teuthology.orchestra.run.smithi082.stdout:(24/28): python3-zc-lockfile-2.0-2.el8.noarch.r 273 kB/s | 23 kB 00:00 2023-10-18T15:56:20.691 INFO:teuthology.orchestra.run.smithi082.stdout:(25/28): python3-markupsafe-0.23-19.el8.x86_64. 941 kB/s | 39 kB 00:00 2023-10-18T15:56:20.695 INFO:teuthology.orchestra.run.smithi082.stdout:(26/28): python3-webtest-2.0.33-1.el8.noarch.rp 492 kB/s | 85 kB 00:00 2023-10-18T15:56:20.730 INFO:teuthology.orchestra.run.smithi082.stdout:(27/28): python3-pytz-2017.2-9.el8.noarch.rpm 1.4 MB/s | 54 kB 00:00 2023-10-18T15:56:20.734 INFO:teuthology.orchestra.run.smithi082.stdout:(28/28): python3-mako-1.0.6-13.el8.noarch.rpm 2.4 MB/s | 157 kB 00:00 2023-10-18T15:56:20.737 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2023-10-18T15:56:20.737 INFO:teuthology.orchestra.run.smithi082.stdout:Total 10 MB/s | 30 MB 00:02 2023-10-18T15:56:20.742 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2023-10-18T15:56:20.779 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2023-10-18T15:56:20.779 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2023-10-18T15:56:21.211 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2023-10-18T15:56:21.215 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2023-10-18T15:56:22.267 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2023-10-18T15:56:22.494 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/28 2023-10-18T15:56:22.699 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/28 2023-10-18T15:56:22.863 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 3/28 2023-10-18T15:56:23.158 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 4/28 2023-10-18T15:56:23.499 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 5/28 2023-10-18T15:56:23.701 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 6/28 2023-10-18T15:56:23.939 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 7/28 2023-10-18T15:56:24.117 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 8/28 2023-10-18T15:56:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 9/28 2023-10-18T15:56:24.463 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 10/28 2023-10-18T15:56:24.653 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 11/28 2023-10-18T15:56:24.828 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 12/28 2023-10-18T15:56:24.964 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 13/28 2023-10-18T15:56:25.162 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 14/28 2023-10-18T15:56:25.345 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 15/28 2023-10-18T15:56:25.522 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 16/28 2023-10-18T15:56:25.745 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-portend-2.6-1.el8.noarch 17/28 2023-10-18T15:56:25.972 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 18/28 2023-10-18T15:56:26.584 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 19/28 2023-10-18T15:56:26.901 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 20/28 2023-10-18T15:56:27.152 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 21/28 2023-10-18T15:56:27.470 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 22/28 2023-10-18T15:56:27.783 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-modules-core-2:16.2.14-75.g84698e74.el8.n 23/28 2023-10-18T15:56:27.947 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-2:16.2.14-75.g84698e74.el8.x86_64 24/28 2023-10-18T15:56:29.450 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-2:16.2.14-75.g84698e74.el8.x86_64 24/28 2023-10-18T15:56:29.636 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-osd-2:16.2.14-75.g84698e74.el8.x86_64 25/28 2023-10-18T15:56:30.063 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-osd-2:16.2.14-75.g84698e74.el8.x86_64 25/28 2023-10-18T15:56:30.224 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mon-2:16.2.14-75.g84698e74.el8.x86_64 26/28 2023-10-18T15:56:30.444 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mon-2:16.2.14-75.g84698e74.el8.x86_64 26/28 2023-10-18T15:56:30.600 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mds-2:16.2.14-75.g84698e74.el8.x86_64 27/28 2023-10-18T15:56:30.618 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mds-2:16.2.14-75.g84698e74.el8.x86_64 27/28 2023-10-18T15:56:30.740 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-2:16.2.14-75.g84698e74.el8.x86_64 28/28 2023-10-18T15:56:31.126 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-2:16.2.14-75.g84698e74.el8.x86_64 28/28 2023-10-18T15:56:31.126 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-2:16.2.14-75.g84698e74.el8.x86_64 1/28 2023-10-18T15:56:31.127 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mds-2:16.2.14-75.g84698e74.el8.x86_64 2/28 2023-10-18T15:56:31.127 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-2:16.2.14-75.g84698e74.el8.x86_64 3/28 2023-10-18T15:56:31.127 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mon-2:16.2.14-75.g84698e74.el8.x86_64 4/28 2023-10-18T15:56:31.127 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-osd-2:16.2.14-75.g84698e74.el8.x86_64 5/28 2023-10-18T15:56:31.127 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-modules-core-2:16.2.14-75.g84698e74.el8.n 6/28 2023-10-18T15:56:31.128 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/28 2023-10-18T15:56:31.128 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/28 2023-10-18T15:56:31.128 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 9/28 2023-10-18T15:56:31.128 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/28 2023-10-18T15:56:31.128 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 11/28 2023-10-18T15:56:31.129 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 12/28 2023-10-18T15:56:31.129 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 13/28 2023-10-18T15:56:31.129 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 14/28 2023-10-18T15:56:31.129 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 15/28 2023-10-18T15:56:31.129 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-portend-2.6-1.el8.noarch 16/28 2023-10-18T15:56:31.130 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 17/28 2023-10-18T15:56:31.130 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 18/28 2023-10-18T15:56:31.130 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 19/28 2023-10-18T15:56:31.130 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 20/28 2023-10-18T15:56:31.130 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 21/28 2023-10-18T15:56:31.131 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 22/28 2023-10-18T15:56:31.131 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 23/28 2023-10-18T15:56:31.131 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 24/28 2023-10-18T15:56:31.131 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 25/28 2023-10-18T15:56:31.131 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 26/28 2023-10-18T15:56:31.131 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 27/28 2023-10-18T15:56:32.629 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 28/28 2023-10-18T15:56:32.630 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:01:28 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:56:34.471 INFO:teuthology.orchestra.run.smithi082.stdout:Installed products updated. 2023-10-18T15:56:34.571 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:56:34.572 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2023-10-18T15:56:34.572 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:56:34.572 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:56:34.572 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:56:34.572 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core-2:16.2.14-75.g84698e74.el8.noarch 2023-10-18T15:56:34.573 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:56:34.573 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:56:34.573 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-18T15:56:34.573 INFO:teuthology.orchestra.run.smithi082.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-18T15:56:34.573 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-18T15:56:34.573 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-18T15:56:34.574 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-18T15:56:34.574 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-18T15:56:34.574 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-18T15:56:34.574 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-18T15:56:34.574 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-18T15:56:34.575 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-18T15:56:34.575 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-18T15:56:34.575 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend-2.6-1.el8.noarch 2023-10-18T15:56:34.575 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-18T15:56:34.575 INFO:teuthology.orchestra.run.smithi082.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-18T15:56:34.576 INFO:teuthology.orchestra.run.smithi082.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-18T15:56:34.576 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-18T15:56:34.576 INFO:teuthology.orchestra.run.smithi082.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-18T15:56:34.576 INFO:teuthology.orchestra.run.smithi082.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-18T15:56:34.577 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-18T15:56:34.577 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-18T15:56:34.577 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-18T15:56:34.577 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-18T15:56:34.577 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:56:34.577 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-10-18T15:56:34.709 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install ceph-base 2023-10-18T15:56:35.076 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:56:35.936 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:01:31 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:56:37.375 INFO:teuthology.orchestra.run.smithi082.stdout:Package ceph-base-2:16.2.14-75.g84698e74.el8.x86_64 is already installed. 2023-10-18T15:56:37.853 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-10-18T15:56:37.854 INFO:teuthology.orchestra.run.smithi082.stdout:Nothing to do. 2023-10-18T15:56:37.854 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-10-18T15:56:37.955 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install cephadm 2023-10-18T15:56:38.315 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:56:39.180 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:01:35 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:56:40.980 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-10-18T15:56:40.981 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-10-18T15:56:40.981 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2023-10-18T15:56:40.981 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-10-18T15:56:40.981 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2023-10-18T15:56:40.982 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm noarch 2:16.2.14-75.g84698e74.el8 ceph-noarch 84 k 2023-10-18T15:56:40.982 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:56:40.982 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2023-10-18T15:56:40.982 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-10-18T15:56:40.982 INFO:teuthology.orchestra.run.smithi082.stdout:Install 1 Package 2023-10-18T15:56:40.983 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:56:40.983 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 84 k 2023-10-18T15:56:40.983 INFO:teuthology.orchestra.run.smithi082.stdout:Installed size: 348 k 2023-10-18T15:56:40.983 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2023-10-18T15:56:41.146 INFO:teuthology.orchestra.run.smithi082.stdout:cephadm-16.2.14-75.g84698e74.el8.noarch.rpm 512 kB/s | 84 kB 00:00 2023-10-18T15:56:41.147 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2023-10-18T15:56:41.147 INFO:teuthology.orchestra.run.smithi082.stdout:Total 508 kB/s | 84 kB 00:00 2023-10-18T15:56:41.148 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2023-10-18T15:56:41.154 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2023-10-18T15:56:41.154 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2023-10-18T15:56:41.156 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2023-10-18T15:56:41.160 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2023-10-18T15:56:41.214 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2023-10-18T15:56:45.006 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephadm-2:16.2.14-75.g84698e74.el8.noarch 1/1 2023-10-18T15:56:45.148 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephadm-2:16.2.14-75.g84698e74.el8.noarch 1/1 2023-10-18T15:56:45.353 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephadm-2:16.2.14-75.g84698e74.el8.noarch 1/1 2023-10-18T15:56:46.349 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephadm-2:16.2.14-75.g84698e74.el8.noarch 1/1 2023-10-18T15:56:46.349 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:01:42 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:56:48.231 INFO:teuthology.orchestra.run.smithi082.stdout:Installed products updated. 2023-10-18T15:56:48.327 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:56:48.328 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2023-10-18T15:56:48.328 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm-2:16.2.14-75.g84698e74.el8.noarch 2023-10-18T15:56:48.328 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:56:48.328 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-10-18T15:56:48.455 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install ceph-immutable-object-cache 2023-10-18T15:56:48.830 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:56:49.716 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:01:45 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:56:51.574 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-10-18T15:56:51.575 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-10-18T15:56:51.575 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repo Size 2023-10-18T15:56:51.575 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-10-18T15:56:51.575 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2023-10-18T15:56:51.575 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache x86_64 2:16.2.14-75.g84698e74.el8 ceph 161 k 2023-10-18T15:56:51.576 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:56:51.576 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2023-10-18T15:56:51.576 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-10-18T15:56:51.576 INFO:teuthology.orchestra.run.smithi082.stdout:Install 1 Package 2023-10-18T15:56:51.576 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:56:51.577 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 161 k 2023-10-18T15:56:51.577 INFO:teuthology.orchestra.run.smithi082.stdout:Installed size: 470 k 2023-10-18T15:56:51.577 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2023-10-18T15:56:51.745 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-immutable-object-cache-16.2.14-75.g84698e7 951 kB/s | 161 kB 00:00 2023-10-18T15:56:51.746 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2023-10-18T15:56:51.746 INFO:teuthology.orchestra.run.smithi082.stdout:Total 943 kB/s | 161 kB 00:00 2023-10-18T15:56:51.747 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2023-10-18T15:56:51.759 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2023-10-18T15:56:51.759 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2023-10-18T15:56:51.811 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2023-10-18T15:56:51.816 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2023-10-18T15:56:51.946 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2023-10-18T15:56:52.111 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-immutable-object-cache-2:16.2.14-75.g84698e74.e 1/1 2023-10-18T15:56:52.503 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.14-75.g84698e74.e 1/1 2023-10-18T15:56:53.554 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-immutable-object-cache-2:16.2.14-75.g84698e74.e 1/1 2023-10-18T15:56:53.554 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:01:49 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:56:55.439 INFO:teuthology.orchestra.run.smithi082.stdout:Installed products updated. 2023-10-18T15:56:55.535 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:56:55.535 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2023-10-18T15:56:55.535 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:56:55.535 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:56:55.536 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-10-18T15:56:55.641 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install ceph-mgr 2023-10-18T15:56:56.006 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:56:56.879 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:01:52 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:56:58.195 INFO:teuthology.orchestra.run.smithi082.stdout:Package ceph-mgr-2:16.2.14-75.g84698e74.el8.x86_64 is already installed. 2023-10-18T15:56:58.680 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-10-18T15:56:58.682 INFO:teuthology.orchestra.run.smithi082.stdout:Nothing to do. 2023-10-18T15:56:58.682 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-10-18T15:56:58.784 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install ceph-mgr-dashboard 2023-10-18T15:56:59.145 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:57:00.006 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:01:56 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:57:01.839 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-10-18T15:57:01.841 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================================== 2023-10-18T15:57:01.841 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2023-10-18T15:57:01.841 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================================== 2023-10-18T15:57:01.841 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2023-10-18T15:57:01.841 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard noarch 2:16.2.14-75.g84698e74.el8 ceph-noarch 3.1 M 2023-10-18T15:57:01.842 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2023-10-18T15:57:01.842 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards noarch 2:16.2.14-75.g84698e74.el8 ceph-noarch 24 k 2023-10-18T15:57:01.842 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts noarch 2:16.2.14-75.g84698e74.el8 ceph-noarch 15 k 2023-10-18T15:57:01.842 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-18T15:57:01.842 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-18T15:57:01.843 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-18T15:57:01.843 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:57:01.843 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2023-10-18T15:57:01.843 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================================== 2023-10-18T15:57:01.843 INFO:teuthology.orchestra.run.smithi082.stdout:Install 6 Packages 2023-10-18T15:57:01.844 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:57:01.844 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 3.4 M 2023-10-18T15:57:01.844 INFO:teuthology.orchestra.run.smithi082.stdout:Installed size: 22 M 2023-10-18T15:57:01.845 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2023-10-18T15:57:02.071 INFO:teuthology.orchestra.run.smithi082.stdout:(1/6): ceph-grafana-dashboards-16.2.14-75.g8469 122 kB/s | 24 kB 00:00 2023-10-18T15:57:02.134 INFO:teuthology.orchestra.run.smithi082.stdout:(2/6): ceph-prometheus-alerts-16.2.14-75.g84698 57 kB/s | 15 kB 00:00 2023-10-18T15:57:02.379 INFO:teuthology.orchestra.run.smithi082.stdout:(3/6): ceph-mgr-dashboard-16.2.14-75.g84698e74. 6.2 MB/s | 3.1 MB 00:00 2023-10-18T15:57:02.382 INFO:teuthology.orchestra.run.smithi082.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 108 kB/s | 34 kB 00:00 2023-10-18T15:57:02.422 INFO:teuthology.orchestra.run.smithi082.stdout:(5/6): python3-jwt-1.6.1-2.el8.noarch.rpm 989 kB/s | 43 kB 00:00 2023-10-18T15:57:02.672 INFO:teuthology.orchestra.run.smithi082.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 365 kB/s | 196 kB 00:00 2023-10-18T15:57:02.674 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2023-10-18T15:57:02.675 INFO:teuthology.orchestra.run.smithi082.stdout:Total 4.1 MB/s | 3.4 MB 00:00 2023-10-18T15:57:02.676 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2023-10-18T15:57:02.685 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2023-10-18T15:57:02.685 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2023-10-18T15:57:02.871 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2023-10-18T15:57:02.875 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2023-10-18T15:57:03.153 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2023-10-18T15:57:03.336 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 1/6 2023-10-18T15:57:03.559 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 2/6 2023-10-18T15:57:03.744 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 3/6 2023-10-18T15:57:03.865 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-prometheus-alerts-2:16.2.14-75.g84698e74.el8.no 4/6 2023-10-18T15:57:04.987 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-grafana-dashboards-2:16.2.14-75.g84698e74.el8.n 5/6 2023-10-18T15:57:05.238 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-dashboard-2:16.2.14-75.g84698e74.el8.noarch 6/6 2023-10-18T15:57:05.516 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.14-75.g84698e74.el8.noarch 6/6 2023-10-18T15:57:05.517 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-grafana-dashboards-2:16.2.14-75.g84698e74.el8.n 1/6 2023-10-18T15:57:05.517 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-dashboard-2:16.2.14-75.g84698e74.el8.noarch 2/6 2023-10-18T15:57:05.517 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-prometheus-alerts-2:16.2.14-75.g84698e74.el8.no 3/6 2023-10-18T15:57:05.517 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/6 2023-10-18T15:57:05.517 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/6 2023-10-18T15:57:06.562 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 6/6 2023-10-18T15:57:06.563 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:02:02 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:57:08.490 INFO:teuthology.orchestra.run.smithi082.stdout:Installed products updated. 2023-10-18T15:57:08.588 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:57:08.588 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2023-10-18T15:57:08.589 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards-2:16.2.14-75.g84698e74.el8.noarch 2023-10-18T15:57:08.589 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard-2:16.2.14-75.g84698e74.el8.noarch 2023-10-18T15:57:08.589 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts-2:16.2.14-75.g84698e74.el8.noarch 2023-10-18T15:57:08.589 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-18T15:57:08.590 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-18T15:57:08.590 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-18T15:57:08.590 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:57:08.590 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-10-18T15:57:08.698 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install ceph-mgr-diskprediction-local 2023-10-18T15:57:09.064 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:57:09.909 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:02:05 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:57:11.796 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-10-18T15:57:11.797 INFO:teuthology.orchestra.run.smithi082.stdout:====================================================================================================================== 2023-10-18T15:57:11.797 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2023-10-18T15:57:11.797 INFO:teuthology.orchestra.run.smithi082.stdout:====================================================================================================================== 2023-10-18T15:57:11.797 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2023-10-18T15:57:11.798 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.14-75.g84698e74.el8 ceph-noarch 8.6 M 2023-10-18T15:57:11.798 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2023-10-18T15:57:11.798 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-18T15:57:11.798 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-18T15:57:11.799 INFO:teuthology.orchestra.run.smithi082.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-18T15:57:11.799 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-18T15:57:11.799 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-18T15:57:11.799 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-18T15:57:11.799 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-18T15:57:11.800 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:57:11.800 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2023-10-18T15:57:11.800 INFO:teuthology.orchestra.run.smithi082.stdout:====================================================================================================================== 2023-10-18T15:57:11.800 INFO:teuthology.orchestra.run.smithi082.stdout:Install 8 Packages 2023-10-18T15:57:11.800 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:57:11.801 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 37 M 2023-10-18T15:57:11.801 INFO:teuthology.orchestra.run.smithi082.stdout:Installed size: 199 M 2023-10-18T15:57:11.801 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2023-10-18T15:57:11.847 INFO:teuthology.orchestra.run.smithi082.stdout:(1/8): python3-numpy-f2py-1.14.3-9.el8.x86_64.r 4.6 MB/s | 225 kB 00:00 2023-10-18T15:57:11.927 INFO:teuthology.orchestra.run.smithi082.stdout:(2/8): python3-numpy-1.14.3-9.el8.x86_64.rpm 47 MB/s | 3.7 MB 00:00 2023-10-18T15:57:12.098 INFO:teuthology.orchestra.run.smithi082.stdout:(3/8): python3-scipy-1.0.0-20.module+el8.1.0+33 49 MB/s | 15 MB 00:00 2023-10-18T15:57:12.120 INFO:teuthology.orchestra.run.smithi082.stdout:(4/8): openblas-threads-0.3.12-1.el8.x86_64.rpm 25 MB/s | 4.7 MB 00:00 2023-10-18T15:57:12.153 INFO:teuthology.orchestra.run.smithi082.stdout:(5/8): libquadmath-8.4.1-1.el8.x86_64.rpm 5.0 MB/s | 169 kB 00:00 2023-10-18T15:57:12.180 INFO:teuthology.orchestra.run.smithi082.stdout:(6/8): openblas-0.3.12-1.el8.x86_64.rpm 56 MB/s | 4.6 MB 00:00 2023-10-18T15:57:12.190 INFO:teuthology.orchestra.run.smithi082.stdout:(7/8): libgfortran-8.4.1-1.el8.x86_64.rpm 17 MB/s | 639 kB 00:00 2023-10-18T15:57:12.542 INFO:teuthology.orchestra.run.smithi082.stdout:(8/8): ceph-mgr-diskprediction-local-16.2.14-75 12 MB/s | 8.6 MB 00:00 2023-10-18T15:57:12.544 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2023-10-18T15:57:12.544 INFO:teuthology.orchestra.run.smithi082.stdout:Total 50 MB/s | 37 MB 00:00 2023-10-18T15:57:12.553 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2023-10-18T15:57:12.572 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2023-10-18T15:57:12.572 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2023-10-18T15:57:12.902 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2023-10-18T15:57:12.906 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2023-10-18T15:57:13.533 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2023-10-18T15:57:13.673 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 1/8 2023-10-18T15:57:13.851 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 1/8 2023-10-18T15:57:13.989 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 2/8 2023-10-18T15:57:14.521 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 2/8 2023-10-18T15:57:14.697 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 3/8 2023-10-18T15:57:15.229 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 3/8 2023-10-18T15:57:15.455 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-0.3.12-1.el8.x86_64 4/8 2023-10-18T15:57:16.233 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 4/8 2023-10-18T15:57:16.539 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 5/8 2023-10-18T15:57:16.763 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2023-10-18T15:57:18.842 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2023-10-18T15:57:20.234 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 7/8 2023-10-18T15:57:20.421 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.14-75.g84698e74 8/8 2023-10-18T15:57:20.720 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.14-75.g84698e74 8/8 2023-10-18T15:57:20.720 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.14-75.g84698e74 1/8 2023-10-18T15:57:20.720 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 2/8 2023-10-18T15:57:20.721 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 3/8 2023-10-18T15:57:20.721 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 4/8 2023-10-18T15:57:20.721 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 5/8 2023-10-18T15:57:20.722 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 6/8 2023-10-18T15:57:20.722 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 7/8 2023-10-18T15:57:21.751 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 8/8 2023-10-18T15:57:21.752 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:02:17 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:57:23.666 INFO:teuthology.orchestra.run.smithi082.stdout:Installed products updated. 2023-10-18T15:57:23.765 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:57:23.766 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2023-10-18T15:57:23.766 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local-2:16.2.14-75.g84698e74.el8.noarch 2023-10-18T15:57:23.766 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-18T15:57:23.766 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-18T15:57:23.767 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-18T15:57:23.767 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-18T15:57:23.767 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-18T15:57:23.767 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-18T15:57:23.767 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-18T15:57:23.768 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:57:23.768 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-10-18T15:57:23.904 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install ceph-mgr-rook 2023-10-18T15:57:24.268 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:57:25.119 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:02:21 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:57:26.938 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-10-18T15:57:26.940 INFO:teuthology.orchestra.run.smithi082.stdout:======================================================================================================= 2023-10-18T15:57:26.940 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2023-10-18T15:57:26.940 INFO:teuthology.orchestra.run.smithi082.stdout:======================================================================================================= 2023-10-18T15:57:26.941 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2023-10-18T15:57:26.941 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook noarch 2:16.2.14-75.g84698e74.el8 ceph-noarch 27 k 2023-10-18T15:57:26.942 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2023-10-18T15:57:26.942 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-18T15:57:26.942 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-18T15:57:26.943 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-18T15:57:26.943 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-18T15:57:26.943 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-18T15:57:26.943 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-18T15:57:26.944 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-18T15:57:26.944 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-18T15:57:26.944 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-18T15:57:26.945 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-18T15:57:26.945 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-18T15:57:26.945 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-18T15:57:26.946 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:57:26.946 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2023-10-18T15:57:26.946 INFO:teuthology.orchestra.run.smithi082.stdout:======================================================================================================= 2023-10-18T15:57:26.947 INFO:teuthology.orchestra.run.smithi082.stdout:Install 13 Packages 2023-10-18T15:57:26.947 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:57:26.948 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 1.9 M 2023-10-18T15:57:26.948 INFO:teuthology.orchestra.run.smithi082.stdout:Installed size: 24 M 2023-10-18T15:57:26.948 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2023-10-18T15:57:27.167 INFO:teuthology.orchestra.run.smithi082.stdout:(1/13): python3-certifi-2018.10.15-7.el8.noarch 245 kB/s | 16 kB 00:00 2023-10-18T15:57:27.173 INFO:teuthology.orchestra.run.smithi082.stdout:(2/13): python3-cachetools-3.1.1-4.el8.noarch.r 452 kB/s | 33 kB 00:00 2023-10-18T15:57:27.210 INFO:teuthology.orchestra.run.smithi082.stdout:(3/13): python3-google-auth-1.1.1-10.el8.noarch 1.8 MB/s | 81 kB 00:00 2023-10-18T15:57:27.215 INFO:teuthology.orchestra.run.smithi082.stdout:(4/13): ceph-mgr-rook-16.2.14-75.g84698e74.el8. 238 kB/s | 27 kB 00:00 2023-10-18T15:57:27.233 INFO:teuthology.orchestra.run.smithi082.stdout:(5/13): python3-rsa-4.9-2.el8.noarch.rpm 2.7 MB/s | 64 kB 00:00 2023-10-18T15:57:27.273 INFO:teuthology.orchestra.run.smithi082.stdout:(6/13): python3-kubernetes-11.0.0-6.el8.noarch. 12 MB/s | 1.2 MB 00:00 2023-10-18T15:57:27.284 INFO:teuthology.orchestra.run.smithi082.stdout:(7/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 524 kB/s | 27 kB 00:00 2023-10-18T15:57:27.292 INFO:teuthology.orchestra.run.smithi082.stdout:(8/13): python3-websocket-client-0.56.0-5.el8.n 796 kB/s | 61 kB 00:00 2023-10-18T15:57:27.320 INFO:teuthology.orchestra.run.smithi082.stdout:(9/13): python3-pyasn1-modules-0.3.7-6.el8.noar 3.0 MB/s | 110 kB 00:00 2023-10-18T15:57:27.323 INFO:teuthology.orchestra.run.smithi082.stdout:(10/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-10-18T15:57:27.343 INFO:teuthology.orchestra.run.smithi082.stdout:(11/13): python3-jsonpointer-1.10-11.el8.noarch 393 kB/s | 20 kB 00:00 2023-10-18T15:57:27.354 INFO:teuthology.orchestra.run.smithi082.stdout:(12/13): python3-oauthlib-2.1.0-1.el8.noarch.rp 4.6 MB/s | 155 kB 00:00 2023-10-18T15:57:27.356 INFO:teuthology.orchestra.run.smithi082.stdout:(13/13): python3-requests-oauthlib-1.0.0-1.el8. 1.2 MB/s | 43 kB 00:00 2023-10-18T15:57:27.357 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2023-10-18T15:57:27.358 INFO:teuthology.orchestra.run.smithi082.stdout:Total 4.6 MB/s | 1.9 MB 00:00 2023-10-18T15:57:27.363 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2023-10-18T15:57:27.377 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2023-10-18T15:57:27.377 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2023-10-18T15:57:27.461 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2023-10-18T15:57:27.466 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2023-10-18T15:57:27.981 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2023-10-18T15:57:28.207 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-10-18T15:57:28.467 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rsa-4.9-2.el8.noarch 2/13 2023-10-18T15:57:28.746 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/13 2023-10-18T15:57:28.994 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 4/13 2023-10-18T15:57:29.251 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 5/13 2023-10-18T15:57:29.426 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 6/13 2023-10-18T15:57:29.621 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 7/13 2023-10-18T15:57:29.835 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 8/13 2023-10-18T15:57:30.069 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-10-18T15:57:30.286 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 10/13 2023-10-18T15:57:31.455 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 11/13 2023-10-18T15:57:32.095 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 12/13 2023-10-18T15:57:32.251 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-rook-2:16.2.14-75.g84698e74.el8.noarch 13/13 2023-10-18T15:57:32.554 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.14-75.g84698e74.el8.noarch 13/13 2023-10-18T15:57:32.554 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-rook-2:16.2.14-75.g84698e74.el8.noarch 1/13 2023-10-18T15:57:32.554 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/13 2023-10-18T15:57:32.555 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/13 2023-10-18T15:57:32.555 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/13 2023-10-18T15:57:32.555 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 5/13 2023-10-18T15:57:32.555 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 6/13 2023-10-18T15:57:32.556 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/13 2023-10-18T15:57:32.556 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 8/13 2023-10-18T15:57:32.556 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 9/13 2023-10-18T15:57:32.556 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 10/13 2023-10-18T15:57:32.556 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-10-18T15:57:32.557 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 12/13 2023-10-18T15:57:33.662 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 13/13 2023-10-18T15:57:33.663 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:02:29 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:57:35.516 INFO:teuthology.orchestra.run.smithi082.stdout:Installed products updated. 2023-10-18T15:57:35.614 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:57:35.614 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2023-10-18T15:57:35.615 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook-2:16.2.14-75.g84698e74.el8.noarch 2023-10-18T15:57:35.615 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-18T15:57:35.615 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-18T15:57:35.615 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-18T15:57:35.615 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-18T15:57:35.616 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-18T15:57:35.616 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-18T15:57:35.616 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-18T15:57:35.617 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-18T15:57:35.617 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-18T15:57:35.617 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-18T15:57:35.617 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-18T15:57:35.618 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-18T15:57:35.618 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:57:35.618 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-10-18T15:57:35.750 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install ceph-mgr-cephadm 2023-10-18T15:57:36.120 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:57:36.991 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:02:32 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:57:38.842 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-10-18T15:57:38.847 INFO:teuthology.orchestra.run.smithi082.stdout:============================================================================================== 2023-10-18T15:57:38.847 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2023-10-18T15:57:38.847 INFO:teuthology.orchestra.run.smithi082.stdout:============================================================================================== 2023-10-18T15:57:38.847 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2023-10-18T15:57:38.848 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm noarch 2:16.2.14-75.g84698e74.el8 ceph-noarch 116 k 2023-10-18T15:57:38.848 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2023-10-18T15:57:38.848 INFO:teuthology.orchestra.run.smithi082.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2023-10-18T15:57:38.848 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-18T15:57:38.848 INFO:teuthology.orchestra.run.smithi082.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2023-10-18T15:57:38.849 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-18T15:57:38.849 INFO:teuthology.orchestra.run.smithi082.stdout: python3-remoto noarch 1.2.1-1.el8 epel 49 k 2023-10-18T15:57:38.849 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:57:38.849 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2023-10-18T15:57:38.849 INFO:teuthology.orchestra.run.smithi082.stdout:============================================================================================== 2023-10-18T15:57:38.850 INFO:teuthology.orchestra.run.smithi082.stdout:Install 6 Packages 2023-10-18T15:57:38.850 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:57:38.850 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 5.8 M 2023-10-18T15:57:38.850 INFO:teuthology.orchestra.run.smithi082.stdout:Installed size: 24 M 2023-10-18T15:57:38.850 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2023-10-18T15:57:39.259 INFO:teuthology.orchestra.run.smithi082.stdout:(1/6): python3-apipkg-1.5-6.el8.noarch.rpm 330 kB/s | 21 kB 00:00 2023-10-18T15:57:39.305 INFO:teuthology.orchestra.run.smithi082.stdout:(2/6): python3-remoto-1.2.1-1.el8.noarch.rpm 1.0 MB/s | 49 kB 00:00 2023-10-18T15:57:39.329 INFO:teuthology.orchestra.run.smithi082.stdout:(3/6): python3-execnet-1.7.1-1.el8.noarch.rpm 2.6 MB/s | 355 kB 00:00 2023-10-18T15:57:39.352 INFO:teuthology.orchestra.run.smithi082.stdout:(4/6): ceph-mgr-cephadm-16.2.14-75.g84698e74.el 739 kB/s | 116 kB 00:00 2023-10-18T15:57:39.400 INFO:teuthology.orchestra.run.smithi082.stdout:(5/6): python3-jinja2-2.10.1-2.el8_0.noarch.rpm 9.7 MB/s | 538 kB 00:00 2023-10-18T15:57:39.401 INFO:teuthology.orchestra.run.smithi082.stdout:(6/6): python3-babel-2.5.1-5.el8.noarch.rpm 52 MB/s | 4.8 MB 00:00 2023-10-18T15:57:39.401 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2023-10-18T15:57:39.401 INFO:teuthology.orchestra.run.smithi082.stdout:Total 11 MB/s | 5.8 MB 00:00 2023-10-18T15:57:39.402 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2023-10-18T15:57:39.412 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2023-10-18T15:57:39.412 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2023-10-18T15:57:39.505 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2023-10-18T15:57:39.602 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2023-10-18T15:57:40.548 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2023-10-18T15:57:40.898 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 1/6 2023-10-18T15:57:41.132 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 2/6 2023-10-18T15:57:41.433 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 3/6 2023-10-18T15:57:41.732 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 4/6 2023-10-18T15:57:41.952 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-remoto-1.2.1-1.el8.noarch 5/6 2023-10-18T15:57:42.099 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-cephadm-2:16.2.14-75.g84698e74.el8.noarch 6/6 2023-10-18T15:57:42.349 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.2.14-75.g84698e74.el8.noarch 6/6 2023-10-18T15:57:42.349 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-cephadm-2:16.2.14-75.g84698e74.el8.noarch 1/6 2023-10-18T15:57:42.349 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 2/6 2023-10-18T15:57:42.349 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 3/6 2023-10-18T15:57:42.350 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-remoto-1.2.1-1.el8.noarch 4/6 2023-10-18T15:57:42.350 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 5/6 2023-10-18T15:57:43.402 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 6/6 2023-10-18T15:57:43.402 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:02:39 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:57:45.296 INFO:teuthology.orchestra.run.smithi082.stdout:Installed products updated. 2023-10-18T15:57:45.396 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:57:45.396 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2023-10-18T15:57:45.397 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm-2:16.2.14-75.g84698e74.el8.noarch 2023-10-18T15:57:45.397 INFO:teuthology.orchestra.run.smithi082.stdout: python3-apipkg-1.5-6.el8.noarch 2023-10-18T15:57:45.397 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-18T15:57:45.397 INFO:teuthology.orchestra.run.smithi082.stdout: python3-execnet-1.7.1-1.el8.noarch 2023-10-18T15:57:45.397 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-18T15:57:45.398 INFO:teuthology.orchestra.run.smithi082.stdout: python3-remoto-1.2.1-1.el8.noarch 2023-10-18T15:57:45.398 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:57:45.398 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-10-18T15:57:45.556 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install ceph-fuse 2023-10-18T15:57:45.930 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:57:46.814 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:02:42 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:57:49.405 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-10-18T15:57:49.407 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-10-18T15:57:49.407 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2023-10-18T15:57:49.407 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-10-18T15:57:49.408 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2023-10-18T15:57:49.408 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse 2023-10-18T15:57:49.408 INFO:teuthology.orchestra.run.smithi082.stdout: x86_64 2:16.2.14-75.g84698e74.el8 ceph 910 k 2023-10-18T15:57:49.408 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2023-10-18T15:57:49.409 INFO:teuthology.orchestra.run.smithi082.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-18T15:57:49.409 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:57:49.409 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2023-10-18T15:57:49.409 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-10-18T15:57:49.410 INFO:teuthology.orchestra.run.smithi082.stdout:Install 2 Packages 2023-10-18T15:57:49.410 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:57:49.410 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 993 k 2023-10-18T15:57:49.410 INFO:teuthology.orchestra.run.smithi082.stdout:Installed size: 2.8 M 2023-10-18T15:57:49.410 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2023-10-18T15:57:49.450 INFO:teuthology.orchestra.run.smithi082.stdout:(1/2): fuse-2.9.7-12.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-10-18T15:57:49.802 INFO:teuthology.orchestra.run.smithi082.stdout:(2/2): ceph-fuse-16.2.14-75.g84698e74.el8.x86_6 2.3 MB/s | 910 kB 00:00 2023-10-18T15:57:49.803 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2023-10-18T15:57:49.804 INFO:teuthology.orchestra.run.smithi082.stdout:Total 2.4 MB/s | 993 kB 00:00 2023-10-18T15:57:49.805 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2023-10-18T15:57:49.826 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2023-10-18T15:57:49.827 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2023-10-18T15:57:49.911 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2023-10-18T15:57:49.916 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2023-10-18T15:57:50.081 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2023-10-18T15:57:50.314 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : fuse-2.9.7-12.el8.x86_64 1/2 2023-10-18T15:57:50.531 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-fuse-2:16.2.14-75.g84698e74.el8.x86_64 2/2 2023-10-18T15:57:50.916 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-fuse-2:16.2.14-75.g84698e74.el8.x86_64 2/2 2023-10-18T15:57:50.916 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-fuse-2:16.2.14-75.g84698e74.el8.x86_64 1/2 2023-10-18T15:57:51.981 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 2/2 2023-10-18T15:57:51.982 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:02:47 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:57:53.847 INFO:teuthology.orchestra.run.smithi082.stdout:Installed products updated. 2023-10-18T15:57:53.945 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:57:53.945 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2023-10-18T15:57:53.945 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse-2:16.2.14-75.g84698e74.el8.x86_64 fuse-2.9.7-12.el8.x86_64 2023-10-18T15:57:53.946 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:57:53.946 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-10-18T15:57:54.058 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install librados-devel 2023-10-18T15:57:54.435 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:57:55.380 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:02:51 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:57:57.256 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-10-18T15:57:57.257 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-10-18T15:57:57.257 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repo Size 2023-10-18T15:57:57.258 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-10-18T15:57:57.258 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2023-10-18T15:57:57.258 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel x86_64 2:16.2.14-75.g84698e74.el8 ceph 131 k 2023-10-18T15:57:57.258 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:57:57.258 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2023-10-18T15:57:57.259 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-10-18T15:57:57.259 INFO:teuthology.orchestra.run.smithi082.stdout:Install 1 Package 2023-10-18T15:57:57.259 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:57:57.259 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 131 k 2023-10-18T15:57:57.259 INFO:teuthology.orchestra.run.smithi082.stdout:Installed size: 432 k 2023-10-18T15:57:57.260 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2023-10-18T15:57:57.433 INFO:teuthology.orchestra.run.smithi082.stdout:librados-devel-16.2.14-75.g84698e74.el8.x86_64. 750 kB/s | 131 kB 00:00 2023-10-18T15:57:57.433 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2023-10-18T15:57:57.434 INFO:teuthology.orchestra.run.smithi082.stdout:Total 744 kB/s | 131 kB 00:00 2023-10-18T15:57:57.434 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2023-10-18T15:57:57.440 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2023-10-18T15:57:57.441 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2023-10-18T15:57:57.498 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2023-10-18T15:57:57.502 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2023-10-18T15:57:57.623 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2023-10-18T15:57:57.803 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librados-devel-2:16.2.14-75.g84698e74.el8.x86_64 1/1 2023-10-18T15:57:58.071 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados-devel-2:16.2.14-75.g84698e74.el8.x86_64 1/1 2023-10-18T15:57:59.103 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados-devel-2:16.2.14-75.g84698e74.el8.x86_64 1/1 2023-10-18T15:57:59.104 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:02:55 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:58:01.005 INFO:teuthology.orchestra.run.smithi082.stdout:Installed products updated. 2023-10-18T15:58:01.103 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:01.103 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2023-10-18T15:58:01.103 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:58:01.103 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:01.104 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-10-18T15:58:01.211 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install libcephfs2 2023-10-18T15:58:01.585 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:58:02.436 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:02:58 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:58:03.822 INFO:teuthology.orchestra.run.smithi082.stdout:Package libcephfs2-2:16.2.14-75.g84698e74.el8.x86_64 is already installed. 2023-10-18T15:58:04.298 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-10-18T15:58:04.299 INFO:teuthology.orchestra.run.smithi082.stdout:Nothing to do. 2023-10-18T15:58:04.300 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-10-18T15:58:04.394 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install libcephfs-devel 2023-10-18T15:58:04.767 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:58:05.661 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:03:01 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:58:07.515 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-10-18T15:58:07.516 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-10-18T15:58:07.516 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repo Size 2023-10-18T15:58:07.516 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-10-18T15:58:07.517 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2023-10-18T15:58:07.517 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel x86_64 2:16.2.14-75.g84698e74.el8 ceph 25 k 2023-10-18T15:58:07.517 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:07.517 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2023-10-18T15:58:07.517 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-10-18T15:58:07.518 INFO:teuthology.orchestra.run.smithi082.stdout:Install 1 Package 2023-10-18T15:58:07.518 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:07.518 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 25 k 2023-10-18T15:58:07.518 INFO:teuthology.orchestra.run.smithi082.stdout:Installed size: 107 k 2023-10-18T15:58:07.518 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2023-10-18T15:58:07.738 INFO:teuthology.orchestra.run.smithi082.stdout:libcephfs-devel-16.2.14-75.g84698e74.el8.x86_64 115 kB/s | 25 kB 00:00 2023-10-18T15:58:07.739 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2023-10-18T15:58:07.739 INFO:teuthology.orchestra.run.smithi082.stdout:Total 115 kB/s | 25 kB 00:00 2023-10-18T15:58:07.739 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2023-10-18T15:58:07.743 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2023-10-18T15:58:07.743 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2023-10-18T15:58:07.745 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2023-10-18T15:58:07.749 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2023-10-18T15:58:07.799 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2023-10-18T15:58:07.951 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs-devel-2:16.2.14-75.g84698e74.el8.x86_64 1/1 2023-10-18T15:58:08.191 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephfs-devel-2:16.2.14-75.g84698e74.el8.x86_64 1/1 2023-10-18T15:58:09.180 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs-devel-2:16.2.14-75.g84698e74.el8.x86_64 1/1 2023-10-18T15:58:09.181 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:03:05 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:58:11.047 INFO:teuthology.orchestra.run.smithi082.stdout:Installed products updated. 2023-10-18T15:58:11.144 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:11.145 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2023-10-18T15:58:11.145 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:58:11.145 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:11.145 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-10-18T15:58:11.250 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install librados2 2023-10-18T15:58:11.624 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:58:12.473 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:03:08 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:58:13.819 INFO:teuthology.orchestra.run.smithi082.stdout:Package librados2-2:16.2.14-75.g84698e74.el8.x86_64 is already installed. 2023-10-18T15:58:14.289 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-10-18T15:58:14.290 INFO:teuthology.orchestra.run.smithi082.stdout:Nothing to do. 2023-10-18T15:58:14.290 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-10-18T15:58:14.392 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install librbd1 2023-10-18T15:58:14.763 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:58:15.669 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:03:11 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:58:17.010 INFO:teuthology.orchestra.run.smithi082.stdout:Package librbd1-2:16.2.14-75.g84698e74.el8.x86_64 is already installed. 2023-10-18T15:58:17.480 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-10-18T15:58:17.481 INFO:teuthology.orchestra.run.smithi082.stdout:Nothing to do. 2023-10-18T15:58:17.481 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-10-18T15:58:17.581 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install python3-rados 2023-10-18T15:58:17.953 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:58:18.894 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:03:14 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:58:20.201 INFO:teuthology.orchestra.run.smithi082.stdout:Package python3-rados-2:16.2.14-75.g84698e74.el8.x86_64 is already installed. 2023-10-18T15:58:20.678 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-10-18T15:58:20.679 INFO:teuthology.orchestra.run.smithi082.stdout:Nothing to do. 2023-10-18T15:58:20.679 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-10-18T15:58:20.803 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install python3-rgw 2023-10-18T15:58:21.177 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:58:22.028 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:03:18 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:58:23.367 INFO:teuthology.orchestra.run.smithi082.stdout:Package python3-rgw-2:16.2.14-75.g84698e74.el8.x86_64 is already installed. 2023-10-18T15:58:23.840 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-10-18T15:58:23.841 INFO:teuthology.orchestra.run.smithi082.stdout:Nothing to do. 2023-10-18T15:58:23.842 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-10-18T15:58:23.947 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install python3-cephfs 2023-10-18T15:58:24.317 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:58:25.176 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:03:21 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:58:26.492 INFO:teuthology.orchestra.run.smithi082.stdout:Package python3-cephfs-2:16.2.14-75.g84698e74.el8.x86_64 is already installed. 2023-10-18T15:58:26.964 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-10-18T15:58:26.965 INFO:teuthology.orchestra.run.smithi082.stdout:Nothing to do. 2023-10-18T15:58:26.965 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-10-18T15:58:27.064 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install python3-rbd 2023-10-18T15:58:27.433 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:58:28.253 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:03:24 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:58:29.574 INFO:teuthology.orchestra.run.smithi082.stdout:Package python3-rbd-2:16.2.14-75.g84698e74.el8.x86_64 is already installed. 2023-10-18T15:58:30.046 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-10-18T15:58:30.047 INFO:teuthology.orchestra.run.smithi082.stdout:Nothing to do. 2023-10-18T15:58:30.048 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-10-18T15:58:30.153 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install rbd-fuse 2023-10-18T15:58:30.527 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:58:31.367 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:03:27 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:58:33.214 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-10-18T15:58:33.215 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-10-18T15:58:33.215 INFO:teuthology.orchestra.run.smithi082.stdout: Package Architecture Version Repository Size 2023-10-18T15:58:33.216 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-10-18T15:58:33.216 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2023-10-18T15:58:33.216 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse x86_64 2:16.2.14-75.g84698e74.el8 ceph 88 k 2023-10-18T15:58:33.216 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:33.216 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2023-10-18T15:58:33.217 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-10-18T15:58:33.217 INFO:teuthology.orchestra.run.smithi082.stdout:Install 1 Package 2023-10-18T15:58:33.217 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:33.217 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 88 k 2023-10-18T15:58:33.217 INFO:teuthology.orchestra.run.smithi082.stdout:Installed size: 240 k 2023-10-18T15:58:33.218 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2023-10-18T15:58:33.795 INFO:teuthology.orchestra.run.smithi082.stdout:rbd-fuse-16.2.14-75.g84698e74.el8.x86_64.rpm 153 kB/s | 88 kB 00:00 2023-10-18T15:58:33.796 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2023-10-18T15:58:33.796 INFO:teuthology.orchestra.run.smithi082.stdout:Total 152 kB/s | 88 kB 00:00 2023-10-18T15:58:33.796 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2023-10-18T15:58:33.810 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2023-10-18T15:58:33.811 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2023-10-18T15:58:33.865 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2023-10-18T15:58:33.870 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2023-10-18T15:58:33.990 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2023-10-18T15:58:34.187 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-fuse-2:16.2.14-75.g84698e74.el8.x86_64 1/1 2023-10-18T15:58:34.463 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: rbd-fuse-2:16.2.14-75.g84698e74.el8.x86_64 1/1 2023-10-18T15:58:35.512 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-fuse-2:16.2.14-75.g84698e74.el8.x86_64 1/1 2023-10-18T15:58:35.512 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:03:31 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:58:37.598 INFO:teuthology.orchestra.run.smithi082.stdout:Installed products updated. 2023-10-18T15:58:37.694 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:37.694 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2023-10-18T15:58:37.695 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:58:37.695 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:37.695 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-10-18T15:58:37.802 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install rbd-mirror 2023-10-18T15:58:38.177 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:58:39.057 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:03:35 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:58:40.843 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-10-18T15:58:40.843 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-10-18T15:58:40.844 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repo Size 2023-10-18T15:58:40.844 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-10-18T15:58:40.844 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2023-10-18T15:58:40.844 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror x86_64 2:16.2.14-75.g84698e74.el8 ceph 3.7 M 2023-10-18T15:58:40.845 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:40.845 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2023-10-18T15:58:40.845 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-10-18T15:58:40.845 INFO:teuthology.orchestra.run.smithi082.stdout:Install 1 Package 2023-10-18T15:58:40.845 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:40.845 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 3.7 M 2023-10-18T15:58:40.846 INFO:teuthology.orchestra.run.smithi082.stdout:Installed size: 15 M 2023-10-18T15:58:40.846 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2023-10-18T15:58:41.336 INFO:teuthology.orchestra.run.smithi082.stdout:rbd-mirror-16.2.14-75.g84698e74.el8.x86_64.rpm 7.5 MB/s | 3.7 MB 00:00 2023-10-18T15:58:41.337 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2023-10-18T15:58:41.338 INFO:teuthology.orchestra.run.smithi082.stdout:Total 7.4 MB/s | 3.7 MB 00:00 2023-10-18T15:58:41.338 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2023-10-18T15:58:41.352 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2023-10-18T15:58:41.352 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2023-10-18T15:58:41.421 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2023-10-18T15:58:41.425 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2023-10-18T15:58:41.917 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2023-10-18T15:58:42.136 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-mirror-2:16.2.14-75.g84698e74.el8.x86_64 1/1 2023-10-18T15:58:42.543 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: rbd-mirror-2:16.2.14-75.g84698e74.el8.x86_64 1/1 2023-10-18T15:58:43.606 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-mirror-2:16.2.14-75.g84698e74.el8.x86_64 1/1 2023-10-18T15:58:43.606 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:03:39 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:58:45.475 INFO:teuthology.orchestra.run.smithi082.stdout:Installed products updated. 2023-10-18T15:58:45.574 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:45.574 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2023-10-18T15:58:45.574 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:58:45.575 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:45.575 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-10-18T15:58:45.695 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install rbd-nbd 2023-10-18T15:58:46.071 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-10-18T15:58:46.924 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:03:42 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:58:48.738 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-10-18T15:58:48.739 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-10-18T15:58:48.739 INFO:teuthology.orchestra.run.smithi082.stdout: Package Architecture Version Repository Size 2023-10-18T15:58:48.739 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-10-18T15:58:48.740 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2023-10-18T15:58:48.740 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd x86_64 2:16.2.14-75.g84698e74.el8 ceph 173 k 2023-10-18T15:58:48.740 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:48.740 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2023-10-18T15:58:48.740 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-10-18T15:58:48.741 INFO:teuthology.orchestra.run.smithi082.stdout:Install 1 Package 2023-10-18T15:58:48.741 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:48.741 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 173 k 2023-10-18T15:58:48.741 INFO:teuthology.orchestra.run.smithi082.stdout:Installed size: 515 k 2023-10-18T15:58:48.741 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2023-10-18T15:58:48.898 INFO:teuthology.orchestra.run.smithi082.stdout:rbd-nbd-16.2.14-75.g84698e74.el8.x86_64.rpm 1.1 MB/s | 173 kB 00:00 2023-10-18T15:58:48.900 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2023-10-18T15:58:48.900 INFO:teuthology.orchestra.run.smithi082.stdout:Total 1.1 MB/s | 173 kB 00:00 2023-10-18T15:58:48.900 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2023-10-18T15:58:48.912 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2023-10-18T15:58:48.912 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2023-10-18T15:58:48.969 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2023-10-18T15:58:48.973 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2023-10-18T15:58:49.097 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2023-10-18T15:58:49.300 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-nbd-2:16.2.14-75.g84698e74.el8.x86_64 1/1 2023-10-18T15:58:49.558 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: rbd-nbd-2:16.2.14-75.g84698e74.el8.x86_64 1/1 2023-10-18T15:58:50.658 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-nbd-2:16.2.14-75.g84698e74.el8.x86_64 1/1 2023-10-18T15:58:50.658 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:03:46 ago on Wed 18 Oct 2023 03:55:04 PM UTC. 2023-10-18T15:58:52.561 INFO:teuthology.orchestra.run.smithi082.stdout:Installed products updated. 2023-10-18T15:58:52.659 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:52.660 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2023-10-18T15:58:52.660 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd-2:16.2.14-75.g84698e74.el8.x86_64 2023-10-18T15:58:52.660 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:52.660 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-10-18T15:58:52.771 DEBUG:teuthology.parallel:result is None 2023-10-18T15:58:52.772 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=84698e743199450ee5f6969d22ebe6bed3a68e4a 2023-10-18T15:58:52.930 DEBUG:teuthology.orchestra.run.smithi082:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-18T15:58:52.950 INFO:teuthology.orchestra.run.smithi082.stdout:16.2.14-75.g84698e74.el8 2023-10-18T15:58:52.951 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-75.g84698e74.el8 2023-10-18T15:58:52.951 INFO:teuthology.task.install:The correct ceph version 16.2.14-75.g84698e74 is installed. 2023-10-18T15:58:52.951 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-18T15:58:52.952 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-10-18T15:58:52.953 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-18T15:58:53.017 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-18T15:58:53.017 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-10-18T15:58:53.018 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/daemon-helper 2023-10-18T15:58:53.082 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-18T15:58:53.146 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-18T15:58:53.146 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-10-18T15:58:53.147 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-18T15:58:53.210 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-18T15:58:53.276 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-18T15:58:53.276 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-10-18T15:58:53.277 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/stdin-killer 2023-10-18T15:58:53.341 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-18T15:58:53.407 INFO:teuthology.run_tasks:Running task ceph... 2023-10-18T15:58:53.700 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-18T15:58:53.700 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 777 /var/log/ceph 2023-10-18T15:58:53.725 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-18T15:58:53.725 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-18T15:58:53.790 INFO:tasks.ceph:Creating extra log directories... 2023-10-18T15:58:53.791 DEBUG:teuthology.orchestra.run.smithi082:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-18T15:58:53.863 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-18T15:58:53.863 INFO:tasks.ceph:config {'conf': {'client': {'rbd default format': 1}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(CACHE_POOL_NO_HIT_SET\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-18T15:58:53.864 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-17_20:38:29-rbd-wip-yuri2-testing-2023-10-11-0853-pacific-distro-default-smithi/7431146', 'branch': 'wip-yuri2-testing-2023-10-11-0853-pacific', 'description': 'rbd/cli_v1/{base/install clusters/{fixed-1 openstack} features/format-1 msgr-failures/few objectstore/bluestore-stupid pool/small-cache-pool supported-random-distro$/{rhel_8} workloads/rbd_cli_generic}', 'email': None, 'first_in_suite': False, 'job_id': '7431146', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-17_20:38:29-rbd-wip-yuri2-testing-2023-10-11-0853-pacific-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 3, 'size': 30}}], 'os_type': 'rhel', 'os_version': '8.4', 'overrides': {'admin_socket': {'branch': 'wip-yuri2-testing-2023-10-11-0853-pacific'}, 'ceph': {'conf': {'client': {'rbd default format': 1}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(CACHE_POOL_NO_HIT_SET\\)'], 'sha1': '84698e743199450ee5f6969d22ebe6bed3a68e4a'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'global': {'osd crush chooseleaf type': 0, 'osd pool default pg num': 128, 'osd pool default pgp num': 128, 'osd pool default size': 2}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '84698e743199450ee5f6969d22ebe6bed3a68e4a'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri2-testing-2023-10-11-0853-pacific', 'sha1': '84698e743199450ee5f6969d22ebe6bed3a68e4a'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'seed': 2214, 'sha1': '84698e743199450ee5f6969d22ebe6bed3a68e4a', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rbd', 'suite_branch': 'wip-yuri2-testing-2023-10-11-0853-pacific', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_84698e743199450ee5f6969d22ebe6bed3a68e4a/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '84698e743199450ee5f6969d22ebe6bed3a68e4a', 'targets': {'smithi082.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOivSv8XGJ5NGv6c6zD4SdEvKnmlMEGAwjHGFVunXQn0rlpezRBKpVUINTonVtsg+ed/sV3IayPLsYHwtPK5Rqw='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'exec': {'client.0': ['sudo ceph osd pool create cache 4', 'sudo ceph osd tier add rbd cache', 'sudo ceph osd tier cache-mode cache writeback', 'sudo ceph osd tier set-overlay rbd cache', 'sudo ceph osd pool set cache hit_set_type bloom', 'sudo ceph osd pool set cache hit_set_count 8', 'sudo ceph osd pool set cache hit_set_period 60', 'sudo ceph osd pool set cache target_max_objects 250']}}, {'workunit': {'clients': {'client.0': ['rbd/cli_generic.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '202b180cb047e798fb131047314a862593f45403', 'timestamp': '2023-10-17_20:38:29', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-18T15:58:53.864 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-18T15:58:53.911 DEBUG:teuthology.orchestra.run.smithi082:> sudo install -d -m0777 -- /var/run/ceph 2023-10-18T15:58:53.975 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-10-18T15:58:53.975 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/scratch_devs of=/dev/stdout 2023-10-18T15:58:54.031 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-18T15:58:54.031 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_1 2023-10-18T15:58:54.085 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-18T15:58:54.086 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T15:58:54.086 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 215821 Links: 1 2023-10-18T15:58:54.086 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T15:58:54.087 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T15:58:54.087 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2023-10-18 15:58:42.310702829 +0000 2023-10-18T15:58:54.087 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2023-10-18 15:54:18.991330878 +0000 2023-10-18T15:58:54.087 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2023-10-18 15:54:18.991330878 +0000 2023-10-18T15:58:54.087 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2023-10-18T15:58:54.088 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-18T15:58:54.148 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2023-10-18T15:58:54.148 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2023-10-18T15:58:54.148 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000237557 s, 2.2 MB/s 2023-10-18T15:58:54.149 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-18T15:58:54.205 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_2 2023-10-18T15:58:54.260 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-18T15:58:54.260 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T15:58:54.261 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 218455 Links: 1 2023-10-18T15:58:54.261 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T15:58:54.261 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T15:58:54.261 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2023-10-18 15:58:42.310702829 +0000 2023-10-18T15:58:54.261 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2023-10-18 15:54:19.483321184 +0000 2023-10-18T15:58:54.262 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2023-10-18 15:54:19.483321184 +0000 2023-10-18T15:58:54.262 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2023-10-18T15:58:54.262 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-18T15:58:54.323 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2023-10-18T15:58:54.324 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2023-10-18T15:58:54.324 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000252354 s, 2.0 MB/s 2023-10-18T15:58:54.325 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-18T15:58:54.381 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_3 2023-10-18T15:58:54.436 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-18T15:58:54.436 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T15:58:54.436 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 223389 Links: 1 2023-10-18T15:58:54.436 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T15:58:54.437 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T15:58:54.437 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2023-10-18 15:58:42.310702829 +0000 2023-10-18T15:58:54.437 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2023-10-18 15:54:19.966311669 +0000 2023-10-18T15:58:54.437 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2023-10-18 15:54:19.966311669 +0000 2023-10-18T15:58:54.437 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2023-10-18T15:58:54.438 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-18T15:58:54.498 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2023-10-18T15:58:54.499 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2023-10-18T15:58:54.499 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000219671 s, 2.3 MB/s 2023-10-18T15:58:54.500 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-18T15:58:54.555 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_4 2023-10-18T15:58:54.611 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-18T15:58:54.611 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T15:58:54.611 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 223451 Links: 1 2023-10-18T15:58:54.611 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T15:58:54.612 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T15:58:54.612 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2023-10-18 15:58:42.310702829 +0000 2023-10-18T15:58:54.612 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2023-10-18 15:54:20.451302113 +0000 2023-10-18T15:58:54.612 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2023-10-18 15:54:20.451302113 +0000 2023-10-18T15:58:54.612 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2023-10-18T15:58:54.613 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-18T15:58:54.673 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2023-10-18T15:58:54.674 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2023-10-18T15:58:54.674 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000193618 s, 2.6 MB/s 2023-10-18T15:58:54.675 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-18T15:58:54.731 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-10-18T15:58:54.732 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi082.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2023-10-18T15:58:54.732 INFO:tasks.ceph:Generating config... 2023-10-18T15:58:54.734 INFO:tasks.ceph:[client] rbd default format = 1 2023-10-18T15:58:54.734 INFO:tasks.ceph:[global] mon client directed command retry = 5 2023-10-18T15:58:54.735 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2023-10-18T15:58:54.735 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-18T15:58:54.735 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-18T15:58:54.736 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-18T15:58:54.736 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-18T15:58:54.736 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-18T15:58:54.737 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-18T15:58:54.737 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-18T15:58:54.737 INFO:tasks.ceph:[osd] bluestore allocator = stupid 2023-10-18T15:58:54.738 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-18T15:58:54.738 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-18T15:58:54.738 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-18T15:58:54.739 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-18T15:58:54.739 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-18T15:58:54.739 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-18T15:58:54.739 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-18T15:58:54.740 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-18T15:58:54.740 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-18T15:58:54.740 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-18T15:58:54.740 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-18T15:58:54.741 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-18T15:58:54.741 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-10-18T15:58:54.741 INFO:tasks.ceph:Setting up mon.a... 2023-10-18T15:58:54.742 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-18T15:58:54.833 INFO:teuthology.orchestra.run.smithi082.stdout:creating /etc/ceph/ceph.keyring 2023-10-18T15:58:54.835 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-18T15:58:54.873 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-18T15:58:54.937 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.82')] 2023-10-18T15:58:54.937 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.82', 'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd default format': 1}, 'mon.a': {}} 2023-10-18T15:58:54.938 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-10-18T15:58:54.939 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-18T15:58:54.991 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.82 --print /home/ubuntu/cephtest/ceph.monmap 2023-10-18T15:58:55.068 INFO:teuthology.orchestra.run.smithi082.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-18T15:58:55.068 INFO:teuthology.orchestra.run.smithi082.stdout:monmaptool: generated fsid cb63dacd-a4a9-467a-b30e-f9493a2182c3 2023-10-18T15:58:55.068 INFO:teuthology.orchestra.run.smithi082.stdout:epoch 0 2023-10-18T15:58:55.068 INFO:teuthology.orchestra.run.smithi082.stdout:fsid cb63dacd-a4a9-467a-b30e-f9493a2182c3 2023-10-18T15:58:55.069 INFO:teuthology.orchestra.run.smithi082.stdout:last_changed 2023-10-18T15:58:55.065525+0000 2023-10-18T15:58:55.069 INFO:teuthology.orchestra.run.smithi082.stdout:created 2023-10-18T15:58:55.065525+0000 2023-10-18T15:58:55.069 INFO:teuthology.orchestra.run.smithi082.stdout:min_mon_release 0 (unknown) 2023-10-18T15:58:55.069 INFO:teuthology.orchestra.run.smithi082.stdout:election_strategy: 1 2023-10-18T15:58:55.070 INFO:teuthology.orchestra.run.smithi082.stdout:0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2023-10-18T15:58:55.070 INFO:teuthology.orchestra.run.smithi082.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2023-10-18T15:58:55.071 DEBUG:teuthology.orchestra.run.smithi082:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-18T15:58:55.125 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID cb63dacd-a4a9-467a-b30e-f9493a2182c3... 2023-10-18T15:58:55.126 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-18T15:58:55.209 INFO:teuthology.orchestra.run.smithi082.stdout:[global] 2023-10-18T15:58:55.209 INFO:teuthology.orchestra.run.smithi082.stdout: chdir = "" 2023-10-18T15:58:55.210 INFO:teuthology.orchestra.run.smithi082.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-18T15:58:55.210 INFO:teuthology.orchestra.run.smithi082.stdout: auth supported = cephx 2023-10-18T15:58:55.210 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:55.210 INFO:teuthology.orchestra.run.smithi082.stdout: filestore xattr use omap = true 2023-10-18T15:58:55.210 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:55.211 INFO:teuthology.orchestra.run.smithi082.stdout: mon clock drift allowed = 1.000 2023-10-18T15:58:55.211 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:55.211 INFO:teuthology.orchestra.run.smithi082.stdout: osd crush chooseleaf type = 0 2023-10-18T15:58:55.211 INFO:teuthology.orchestra.run.smithi082.stdout: auth debug = true 2023-10-18T15:58:55.211 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:55.212 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on old message = true 2023-10-18T15:58:55.212 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on bug = true 2023-10-18T15:58:55.212 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:55.212 INFO:teuthology.orchestra.run.smithi082.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-18T15:58:55.212 INFO:teuthology.orchestra.run.smithi082.stdout: mon pg warn max object skew = 0 2023-10-18T15:58:55.213 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:55.213 INFO:teuthology.orchestra.run.smithi082.stdout: # disable pg_autoscaler by default for new pools 2023-10-18T15:58:55.213 INFO:teuthology.orchestra.run.smithi082.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-18T15:58:55.213 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:55.213 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default size = 2 2023-10-18T15:58:55.213 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:55.214 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow primary affinity = true 2023-10-18T15:58:55.214 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow pg remap = true 2023-10-18T15:58:55.214 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on legacy crush tunables = false 2023-10-18T15:58:55.214 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on crush straw calc version zero = false 2023-10-18T15:58:55.214 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on no sortbitwise = false 2023-10-18T15:58:55.215 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on osd down out interval zero = false 2023-10-18T15:58:55.215 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on too few osds = false 2023-10-18T15:58:55.215 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-18T15:58:55.215 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-18T15:58:55.215 INFO:teuthology.orchestra.run.smithi082.stdout: mon_allow_pool_size_one = true 2023-10-18T15:58:55.216 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:55.216 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2023-10-18T15:58:55.216 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:55.216 INFO:teuthology.orchestra.run.smithi082.stdout: osd default data pool replay window = 5 2023-10-18T15:58:55.217 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:55.217 INFO:teuthology.orchestra.run.smithi082.stdout: mon allow pool delete = true 2023-10-18T15:58:55.217 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:55.217 INFO:teuthology.orchestra.run.smithi082.stdout: mon cluster log file level = debug 2023-10-18T15:58:55.217 INFO:teuthology.orchestra.run.smithi082.stdout: debug asserts on shutdown = true 2023-10-18T15:58:55.218 INFO:teuthology.orchestra.run.smithi082.stdout: mon health detail to clog = false 2023-10-18T15:58:55.218 INFO:teuthology.orchestra.run.smithi082.stdout: mon host = 172.21.15.82 2023-10-18T15:58:55.218 INFO:teuthology.orchestra.run.smithi082.stdout: mon client directed command retry = 5 2023-10-18T15:58:55.218 INFO:teuthology.orchestra.run.smithi082.stdout: ms inject socket failures = 5000 2023-10-18T15:58:55.218 INFO:teuthology.orchestra.run.smithi082.stdout: fsid = cb63dacd-a4a9-467a-b30e-f9493a2182c3 2023-10-18T15:58:55.219 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:55.219 INFO:teuthology.orchestra.run.smithi082.stdout:[osd] 2023-10-18T15:58:55.219 INFO:teuthology.orchestra.run.smithi082.stdout: osd journal size = 100 2023-10-18T15:58:55.219 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:55.219 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub load threshold = 5.0 2023-10-18T15:58:55.219 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub max interval = 600 2023-10-18T15:58:55.220 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:55.220 INFO:teuthology.orchestra.run.smithi082.stdout: osd recover clone overlap = true 2023-10-18T15:58:55.220 INFO:teuthology.orchestra.run.smithi082.stdout: osd recovery max chunk = 1048576 2023-10-18T15:58:55.220 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:55.220 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug shutdown = true 2023-10-18T15:58:55.221 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug op order = true 2023-10-18T15:58:55.221 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug verify stray on activate = true 2023-10-18T15:58:55.221 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:55.221 INFO:teuthology.orchestra.run.smithi082.stdout: osd open classes on start = true 2023-10-18T15:58:55.221 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug pg log writeout = true 2023-10-18T15:58:55.221 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:55.222 INFO:teuthology.orchestra.run.smithi082.stdout: osd deep scrub update digest min age = 30 2023-10-18T15:58:55.222 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:55.222 INFO:teuthology.orchestra.run.smithi082.stdout: osd map max advance = 10 2023-10-18T15:58:55.222 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:55.225 INFO:teuthology.orchestra.run.smithi082.stdout: journal zero on create = true 2023-10-18T15:58:55.225 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:55.226 INFO:teuthology.orchestra.run.smithi082.stdout: filestore ondisk finisher threads = 3 2023-10-18T15:58:55.226 INFO:teuthology.orchestra.run.smithi082.stdout: filestore apply finisher threads = 3 2023-10-18T15:58:55.226 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:55.226 INFO:teuthology.orchestra.run.smithi082.stdout: bdev debug aio = true 2023-10-18T15:58:55.226 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug misdirected ops = true 2023-10-18T15:58:55.227 INFO:teuthology.orchestra.run.smithi082.stdout: bdev async discard = True 2023-10-18T15:58:55.227 INFO:teuthology.orchestra.run.smithi082.stdout: bdev enable discard = True 2023-10-18T15:58:55.227 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore allocator = stupid 2023-10-18T15:58:55.227 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore block size = 96636764160 2023-10-18T15:58:55.227 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore fsck on mount = True 2023-10-18T15:58:55.228 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluefs = 1/20 2023-10-18T15:58:55.228 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluestore = 1/20 2023-10-18T15:58:55.228 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2023-10-18T15:58:55.228 INFO:teuthology.orchestra.run.smithi082.stdout: debug osd = 20 2023-10-18T15:58:55.228 INFO:teuthology.orchestra.run.smithi082.stdout: debug rocksdb = 4/10 2023-10-18T15:58:55.228 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd backfillfull_ratio = 0.85 2023-10-18T15:58:55.229 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd full ratio = 0.9 2023-10-18T15:58:55.229 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd nearfull ratio = 0.8 2023-10-18T15:58:55.229 INFO:teuthology.orchestra.run.smithi082.stdout: osd failsafe full ratio = 0.95 2023-10-18T15:58:55.229 INFO:teuthology.orchestra.run.smithi082.stdout: osd objectstore = bluestore 2023-10-18T15:58:55.229 INFO:teuthology.orchestra.run.smithi082.stdout: osd shutdown pgref assert = True 2023-10-18T15:58:55.230 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:55.230 INFO:teuthology.orchestra.run.smithi082.stdout:[mgr] 2023-10-18T15:58:55.230 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2023-10-18T15:58:55.230 INFO:teuthology.orchestra.run.smithi082.stdout: debug mgr = 20 2023-10-18T15:58:55.230 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2023-10-18T15:58:55.231 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2023-10-18T15:58:55.231 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2023-10-18T15:58:55.231 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2023-10-18T15:58:55.231 INFO:teuthology.orchestra.run.smithi082.stdout: mgr/telemetry/nag = false 2023-10-18T15:58:55.231 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:55.232 INFO:teuthology.orchestra.run.smithi082.stdout:[mon] 2023-10-18T15:58:55.232 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2023-10-18T15:58:55.232 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2023-10-18T15:58:55.232 INFO:teuthology.orchestra.run.smithi082.stdout: debug paxos = 20 2023-10-18T15:58:55.232 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2023-10-18T15:58:55.233 INFO:teuthology.orchestra.run.smithi082.stdout: mon data avail warn = 5 2023-10-18T15:58:55.233 INFO:teuthology.orchestra.run.smithi082.stdout: mon mgr mkfs grace = 240 2023-10-18T15:58:55.233 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2023-10-18T15:58:55.233 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd reporter subtree level = osd 2023-10-18T15:58:55.233 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd prime pg temp = true 2023-10-18T15:58:55.233 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2023-10-18T15:58:55.234 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:55.234 INFO:teuthology.orchestra.run.smithi082.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-18T15:58:55.234 INFO:teuthology.orchestra.run.smithi082.stdout: auth mon ticket ttl = 660 # 11m 2023-10-18T15:58:55.234 INFO:teuthology.orchestra.run.smithi082.stdout: auth service ticket ttl = 240 # 4m 2023-10-18T15:58:55.234 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:55.235 INFO:teuthology.orchestra.run.smithi082.stdout: # don't complain about insecure global_id in the test suite 2023-10-18T15:58:55.235 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-18T15:58:55.235 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-18T15:58:55.235 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:58:55.235 INFO:teuthology.orchestra.run.smithi082.stdout:[client] 2023-10-18T15:58:55.235 INFO:teuthology.orchestra.run.smithi082.stdout: rgw cache enabled = true 2023-10-18T15:58:55.236 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable ops log = true 2023-10-18T15:58:55.236 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable usage log = true 2023-10-18T15:58:55.236 INFO:teuthology.orchestra.run.smithi082.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-18T15:58:55.236 INFO:teuthology.orchestra.run.smithi082.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-18T15:58:55.236 INFO:teuthology.orchestra.run.smithi082.stdout: rbd default format = 1 2023-10-18T15:58:55.237 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.a] 2023-10-18T15:58:55.238 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-18T15:58:55.238 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-18T15:58:55.328 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-18T15:58:55.329 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-10-18T15:58:55.329 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-18T15:58:55.344 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-10-18T15:58:55.344 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-18T15:58:55.398 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi082.front.sepia.ceph.com 2023-10-18T15:58:55.399 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-10-18T15:58:55.399 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-18T15:58:55.399 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-18T15:58:55.472 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-10-18T15:58:55.473 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-18T15:58:55.528 INFO:tasks.ceph:Setting up mon nodes... 2023-10-18T15:58:55.528 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-18T15:58:55.528 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-18T15:58:55.614 INFO:teuthology.orchestra.run.smithi082.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-18T15:58:55.616 INFO:tasks.ceph:Setting up mds nodes... 2023-10-18T15:58:55.617 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-18T15:58:55.617 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-18T15:58:55.654 INFO:teuthology.orchestra.run.smithi082.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-18T15:58:55.666 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-18T15:58:55.667 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi082.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2023-10-18T15:58:55.667 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-18T15:58:55.731 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-10-18T15:58:55.731 INFO:tasks.ceph:role: osd.0 2023-10-18T15:58:55.731 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi082.front.sepia.ceph.com 2023-10-18T15:58:55.732 DEBUG:teuthology.orchestra.run.smithi082:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-18T15:58:55.804 INFO:teuthology.orchestra.run.smithi082.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T15:58:55.805 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T15:58:55.805 INFO:teuthology.orchestra.run.smithi082.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T15:58:55.805 INFO:teuthology.orchestra.run.smithi082.stdout: = reflink=1 2023-10-18T15:58:55.805 INFO:teuthology.orchestra.run.smithi082.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T15:58:55.805 INFO:teuthology.orchestra.run.smithi082.stdout: = sunit=0 swidth=0 blks 2023-10-18T15:58:55.806 INFO:teuthology.orchestra.run.smithi082.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T15:58:55.806 INFO:teuthology.orchestra.run.smithi082.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-18T15:58:55.806 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T15:58:55.807 INFO:teuthology.orchestra.run.smithi082.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T15:58:56.233 INFO:teuthology.orchestra.run.smithi082.stdout:Discarding blocks...Done. 2023-10-18T15:58:56.235 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi082.front.sepia.ceph.com -o noatime 2023-10-18T15:58:56.235 DEBUG:teuthology.orchestra.run.smithi082:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-18T15:58:56.424 DEBUG:teuthology.orchestra.run.smithi082:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-18T15:58:56.455 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-18T15:58:56.521 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-10-18T15:58:56.521 INFO:tasks.ceph:role: osd.1 2023-10-18T15:58:56.521 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi082.front.sepia.ceph.com 2023-10-18T15:58:56.522 DEBUG:teuthology.orchestra.run.smithi082:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-18T15:58:56.592 INFO:teuthology.orchestra.run.smithi082.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T15:58:56.592 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T15:58:56.593 INFO:teuthology.orchestra.run.smithi082.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T15:58:56.593 INFO:teuthology.orchestra.run.smithi082.stdout: = reflink=1 2023-10-18T15:58:56.593 INFO:teuthology.orchestra.run.smithi082.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T15:58:56.593 INFO:teuthology.orchestra.run.smithi082.stdout: = sunit=0 swidth=0 blks 2023-10-18T15:58:56.594 INFO:teuthology.orchestra.run.smithi082.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T15:58:56.594 INFO:teuthology.orchestra.run.smithi082.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-18T15:58:56.594 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T15:58:56.594 INFO:teuthology.orchestra.run.smithi082.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T15:58:57.022 INFO:teuthology.orchestra.run.smithi082.stdout:Discarding blocks...Done. 2023-10-18T15:58:57.024 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi082.front.sepia.ceph.com -o noatime 2023-10-18T15:58:57.024 DEBUG:teuthology.orchestra.run.smithi082:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-18T15:58:57.135 DEBUG:teuthology.orchestra.run.smithi082:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-18T15:58:57.163 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-18T15:58:57.228 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-10-18T15:58:57.229 INFO:tasks.ceph:role: osd.2 2023-10-18T15:58:57.229 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi082.front.sepia.ceph.com 2023-10-18T15:58:57.229 DEBUG:teuthology.orchestra.run.smithi082:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-18T15:58:57.300 INFO:teuthology.orchestra.run.smithi082.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T15:58:57.301 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T15:58:57.301 INFO:teuthology.orchestra.run.smithi082.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T15:58:57.301 INFO:teuthology.orchestra.run.smithi082.stdout: = reflink=1 2023-10-18T15:58:57.301 INFO:teuthology.orchestra.run.smithi082.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T15:58:57.302 INFO:teuthology.orchestra.run.smithi082.stdout: = sunit=0 swidth=0 blks 2023-10-18T15:58:57.302 INFO:teuthology.orchestra.run.smithi082.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T15:58:57.302 INFO:teuthology.orchestra.run.smithi082.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-18T15:58:57.302 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T15:58:57.302 INFO:teuthology.orchestra.run.smithi082.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T15:58:57.744 INFO:teuthology.orchestra.run.smithi082.stdout:Discarding blocks...Done. 2023-10-18T15:58:57.746 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi082.front.sepia.ceph.com -o noatime 2023-10-18T15:58:57.746 DEBUG:teuthology.orchestra.run.smithi082:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-18T15:58:57.895 DEBUG:teuthology.orchestra.run.smithi082:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-18T15:58:57.924 DEBUG:teuthology.orchestra.run.smithi082:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T15:58:58.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-10-18T15:58:58.015+0000 7f2713a7e200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-18T15:58:58.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-10-18T15:58:58.015+0000 7f2713a7e200 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-18T15:58:58.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-10-18T15:58:58.015+0000 7f2713a7e200 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-18T15:58:58.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-10-18T15:58:58.015+0000 7f2713a7e200 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-18T15:58:58.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-10-18T15:58:58.015+0000 7f2713a7e200 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-18T15:58:58.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-10-18T15:58:58.024+0000 7f2713a7e200 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-18T15:59:00.871 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-18T15:59:00.897 DEBUG:teuthology.orchestra.run.smithi082:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T15:59:00.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-10-18T15:59:00.978+0000 7fcebe403200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-18T15:59:00.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-10-18T15:59:00.978+0000 7fcebe403200 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-18T15:59:00.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-10-18T15:59:00.978+0000 7fcebe403200 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-18T15:59:00.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-10-18T15:59:00.978+0000 7fcebe403200 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-18T15:59:00.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-10-18T15:59:00.978+0000 7fcebe403200 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-18T15:59:00.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-10-18T15:59:00.979+0000 7fcebe403200 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-18T15:59:03.826 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-18T15:59:03.893 DEBUG:teuthology.orchestra.run.smithi082:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T15:59:03.977 INFO:teuthology.orchestra.run.smithi082.stderr:2023-10-18T15:59:03.974+0000 7fcf16f1f200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-18T15:59:03.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-10-18T15:59:03.974+0000 7fcf16f1f200 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-18T15:59:03.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-10-18T15:59:03.974+0000 7fcf16f1f200 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-18T15:59:03.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-10-18T15:59:03.974+0000 7fcf16f1f200 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-18T15:59:03.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-10-18T15:59:03.974+0000 7fcf16f1f200 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-18T15:59:03.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-10-18T15:59:03.975+0000 7fcf16f1f200 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-18T15:59:06.822 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-18T15:59:06.891 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-18T15:59:06.892 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-10-18T15:59:06.892 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-18T15:59:06.959 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-10-18T15:59:06.960 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-18T15:59:07.026 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-10-18T15:59:07.026 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-18T15:59:07.092 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-10-18T15:59:07.092 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-18T15:59:07.157 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-10-18T15:59:07.157 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-18T15:59:07.212 INFO:tasks.ceph:Adding keys to all mons... 2023-10-18T15:59:07.212 DEBUG:teuthology.orchestra.run.smithi082:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-18T15:59:07.275 INFO:teuthology.orchestra.run.smithi082.stdout:[mgr.x] 2023-10-18T15:59:07.276 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQC/ADBlv/l6JBAAeQGYB8Bk45BYinXv9IBb5w== 2023-10-18T15:59:07.276 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.0] 2023-10-18T15:59:07.276 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDCADBl77D8ABAAhZs/xbfWoOVsjlAgsaDoPA== 2023-10-18T15:59:07.276 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.1] 2023-10-18T15:59:07.277 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDEADBlCWhROhAAoGr98KVZ3UBjcu6eGlqtMg== 2023-10-18T15:59:07.277 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.2] 2023-10-18T15:59:07.277 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDHADBlIoEXOhAAsbDcGos1YUtBJBOC+onkRQ== 2023-10-18T15:59:07.277 INFO:teuthology.orchestra.run.smithi082.stdout:[client.0] 2023-10-18T15:59:07.277 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQC/ADBlDdfbJhAAeh49j6OMela4MKX2x1zsnQ== 2023-10-18T15:59:07.278 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-18T15:59:07.366 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T15:59:07.413 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T15:59:07.459 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T15:59:07.506 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-18T15:59:07.555 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-18T15:59:07.556 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-18T15:59:07.580 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-18T15:59:07.670 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-18T15:59:07.695 DEBUG:teuthology.orchestra.run.smithi082:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-18T15:59:07.750 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-18T15:59:07.750 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-18T15:59:07.751 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-18T15:59:07.792 INFO:tasks.ceph.mon.a:Started 2023-10-18T15:59:07.792 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-18T15:59:07.793 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-18T15:59:07.793 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-18T15:59:07.795 INFO:tasks.ceph.mgr.x:Started 2023-10-18T15:59:07.795 INFO:tasks.ceph:Setting crush tunables to default 2023-10-18T15:59:07.796 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph osd crush tunables default 2023-10-18T15:59:08.002 INFO:teuthology.orchestra.run.smithi082.stderr:adjusted tunables profile to default 2023-10-18T15:59:08.013 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-18T15:59:08.014 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-10-18T15:59:08.014 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-18T15:59:08.038 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-10-18T15:59:08.039 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-18T15:59:08.103 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-10-18T15:59:08.103 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-18T15:59:08.167 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph osd new f646c787-fe92-4e2d-abce-6b81d0e6e284 0 2023-10-18T15:59:08.190 INFO:tasks.ceph.mgr.x.smithi082.stderr:2023-10-18T15:59:08.187+0000 7f760b0e7680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T15:59:08.262 INFO:tasks.ceph.mgr.x.smithi082.stderr:2023-10-18T15:59:08.259+0000 7f760b0e7680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T15:59:08.391 INFO:teuthology.orchestra.run.smithi082.stdout:0 2023-10-18T15:59:08.403 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph osd new 46810abb-ff37-4aeb-b272-7610274b1957 1 2023-10-18T15:59:08.604 INFO:teuthology.orchestra.run.smithi082.stdout:1 2023-10-18T15:59:08.616 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph osd new a5410593-60ce-4cb3-901e-e96645dc2c4e 2 2023-10-18T15:59:08.774 INFO:tasks.ceph.mgr.x.smithi082.stderr:2023-10-18T15:59:08.771+0000 7f760b0e7680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T15:59:08.816 INFO:teuthology.orchestra.run.smithi082.stdout:2 2023-10-18T15:59:08.829 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-18T15:59:08.829 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-18T15:59:08.831 INFO:tasks.ceph.osd.0:Started 2023-10-18T15:59:08.831 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-18T15:59:08.832 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-18T15:59:08.834 INFO:tasks.ceph.osd.1:Started 2023-10-18T15:59:08.834 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-18T15:59:08.834 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-18T15:59:08.836 INFO:tasks.ceph.osd.2:Started 2023-10-18T15:59:08.837 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T15:59:09.030 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:59:09.031 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":5,"fsid":"cb63dacd-a4a9-467a-b30e-f9493a2182c3","created":"2023-10-18T15:59:07.909789+0000","modified":"2023-10-18T15:59:08.812513+0000","last_up_change":"0.000000","last_in_change":"2023-10-18T15:59:08.812513+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"f646c787-fe92-4e2d-abce-6b81d0e6e284","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"46810abb-ff37-4aeb-b272-7610274b1957","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a5410593-60ce-4cb3-901e-e96645dc2c4e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T15:59:09.043 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-18T15:59:09.043 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-18T15:59:09.345 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-18T15:59:09.426 INFO:tasks.ceph.osd.2.smithi082.stderr:2023-10-18T15:59:09.423+0000 7f2cd99bf200 -1 Falling back to public interface 2023-10-18T15:59:09.430 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T15:59:09.427+0000 7f166c849200 -1 Falling back to public interface 2023-10-18T15:59:09.447 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T15:59:09.445+0000 7fa317742200 -1 Falling back to public interface 2023-10-18T15:59:09.545 INFO:tasks.ceph.mgr.x.smithi082.stderr:2023-10-18T15:59:09.542+0000 7f760b0e7680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T15:59:09.611 INFO:tasks.ceph.mgr.x.smithi082.stderr:2023-10-18T15:59:09.608+0000 7f760b0e7680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T15:59:09.748 INFO:teuthology.misc.health.smithi082.stdout: 2023-10-18T15:59:09.749 INFO:teuthology.misc.health.smithi082.stdout:{"epoch":5,"fsid":"cb63dacd-a4a9-467a-b30e-f9493a2182c3","created":"2023-10-18T15:59:07.909789+0000","modified":"2023-10-18T15:59:08.812513+0000","last_up_change":"0.000000","last_in_change":"2023-10-18T15:59:08.812513+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"f646c787-fe92-4e2d-abce-6b81d0e6e284","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"46810abb-ff37-4aeb-b272-7610274b1957","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a5410593-60ce-4cb3-901e-e96645dc2c4e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T15:59:09.759 DEBUG:teuthology.misc:0 of 3 OSDs are up 2023-10-18T15:59:09.814 INFO:tasks.ceph.mgr.x.smithi082.stderr:2023-10-18T15:59:09.811+0000 7f760b0e7680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T15:59:10.252 INFO:tasks.ceph.mgr.x.smithi082.stderr:2023-10-18T15:59:10.249+0000 7f760b0e7680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T15:59:10.355 INFO:tasks.ceph.mgr.x.smithi082.stderr:2023-10-18T15:59:10.352+0000 7f760b0e7680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T15:59:10.506 INFO:tasks.ceph.mgr.x.smithi082.stderr:2023-10-18T15:59:10.503+0000 7f760b0e7680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T15:59:10.584 INFO:tasks.ceph.mgr.x.smithi082.stderr:2023-10-18T15:59:10.581+0000 7f760b0e7680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T15:59:10.653 INFO:tasks.ceph.mgr.x.smithi082.stderr:2023-10-18T15:59:10.650+0000 7f760b0e7680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T15:59:10.916 INFO:tasks.ceph.mgr.x.smithi082.stderr:2023-10-18T15:59:10.913+0000 7f760b0e7680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T15:59:11.122 INFO:tasks.ceph.mgr.x.smithi082.stderr:2023-10-18T15:59:11.119+0000 7f760b0e7680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T15:59:11.780 INFO:tasks.ceph.mgr.x.smithi082.stderr:2023-10-18T15:59:11.777+0000 7f760b0e7680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T15:59:11.847 INFO:tasks.ceph.mgr.x.smithi082.stderr:2023-10-18T15:59:11.845+0000 7f760b0e7680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T15:59:11.928 INFO:tasks.ceph.mgr.x.smithi082.stderr:2023-10-18T15:59:11.925+0000 7f760b0e7680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T15:59:12.076 INFO:tasks.ceph.mgr.x.smithi082.stderr:2023-10-18T15:59:12.073+0000 7f760b0e7680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T15:59:12.146 INFO:tasks.ceph.mgr.x.smithi082.stderr:2023-10-18T15:59:12.143+0000 7f760b0e7680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T15:59:12.308 INFO:tasks.ceph.mgr.x.smithi082.stderr:2023-10-18T15:59:12.305+0000 7f760b0e7680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T15:59:12.420 INFO:tasks.ceph.mgr.x.smithi082.stderr:2023-10-18T15:59:12.417+0000 7f760b0e7680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T15:59:12.747 INFO:tasks.ceph.mgr.x.smithi082.stderr:2023-10-18T15:59:12.744+0000 7f760b0e7680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T15:59:12.817 INFO:tasks.ceph.mgr.x.smithi082.stderr:2023-10-18T15:59:12.814+0000 7f760b0e7680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T15:59:12.946 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T15:59:12.941+0000 7f166c849200 -1 osd.1 0 log_to_monitors {default=true} 2023-10-18T15:59:12.957 INFO:tasks.ceph.osd.2.smithi082.stderr:2023-10-18T15:59:12.954+0000 7f2cd99bf200 -1 osd.2 0 log_to_monitors {default=true} 2023-10-18T15:59:13.006 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T15:59:13.002+0000 7fa317742200 -1 osd.0 0 log_to_monitors {default=true} 2023-10-18T15:59:13.835 INFO:tasks.ceph.mgr.x.smithi082.stderr:2023-10-18T15:59:13.832+0000 7f75c130b700 -1 mgr.server handle_report got status from non-daemon mon.a 2023-10-18T15:59:14.892 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T15:59:14.889+0000 7f1663206700 -1 osd.1 0 waiting for initial osdmap 2023-10-18T15:59:14.893 INFO:tasks.ceph.osd.2.smithi082.stderr:2023-10-18T15:59:14.889+0000 7f2cd037c700 -1 osd.2 0 waiting for initial osdmap 2023-10-18T15:59:14.893 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T15:59:14.889+0000 7fa30e0ff700 -1 osd.0 0 waiting for initial osdmap 2023-10-18T15:59:14.896 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T15:59:14.893+0000 7f165f982700 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T15:59:14.896 INFO:tasks.ceph.osd.2.smithi082.stderr:2023-10-18T15:59:14.893+0000 7f2cccaf8700 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T15:59:14.898 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T15:59:14.895+0000 7fa30a87b700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T15:59:16.062 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-18T15:59:16.350 INFO:teuthology.misc.health.smithi082.stdout: 2023-10-18T15:59:16.350 INFO:teuthology.misc.health.smithi082.stdout:{"epoch":8,"fsid":"cb63dacd-a4a9-467a-b30e-f9493a2182c3","created":"2023-10-18T15:59:07.909789+0000","modified":"2023-10-18T15:59:15.884655+0000","last_up_change":"2023-10-18T15:59:15.884655+0000","last_in_change":"2023-10-18T15:59:08.812513+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"f646c787-fe92-4e2d-abce-6b81d0e6e284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":87291},{"type":"v1","addr":"172.21.15.82:6817","nonce":87291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":87291},{"type":"v1","addr":"172.21.15.82:6819","nonce":87291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":87291},{"type":"v1","addr":"172.21.15.82:6823","nonce":87291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":87291},{"type":"v1","addr":"172.21.15.82:6821","nonce":87291}]},"public_addr":"172.21.15.82:6817/87291","cluster_addr":"172.21.15.82:6819/87291","heartbeat_back_addr":"172.21.15.82:6823/87291","heartbeat_front_addr":"172.21.15.82:6821/87291","state":["exists","up"]},{"osd":1,"uuid":"46810abb-ff37-4aeb-b272-7610274b1957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":87261},{"type":"v1","addr":"172.21.15.82:6809","nonce":87261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":87261},{"type":"v1","addr":"172.21.15.82:6811","nonce":87261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":87261},{"type":"v1","addr":"172.21.15.82:6815","nonce":87261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":87261},{"type":"v1","addr":"172.21.15.82:6813","nonce":87261}]},"public_addr":"172.21.15.82:6809/87261","cluster_addr":"172.21.15.82:6811/87261","heartbeat_back_addr":"172.21.15.82:6815/87261","heartbeat_front_addr":"172.21.15.82:6813/87261","state":["exists","up"]},{"osd":2,"uuid":"a5410593-60ce-4cb3-901e-e96645dc2c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":87262},{"type":"v1","addr":"172.21.15.82:6801","nonce":87262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":87262},{"type":"v1","addr":"172.21.15.82:6803","nonce":87262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":87262},{"type":"v1","addr":"172.21.15.82:6807","nonce":87262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":87262},{"type":"v1","addr":"172.21.15.82:6805","nonce":87262}]},"public_addr":"172.21.15.82:6801/87262","cluster_addr":"172.21.15.82:6803/87262","heartbeat_back_addr":"172.21.15.82:6807/87262","heartbeat_front_addr":"172.21.15.82:6805/87262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T15:59:16.362 DEBUG:teuthology.misc:3 of 3 OSDs are up 2023-10-18T15:59:16.363 INFO:tasks.ceph:Creating RBD pool 2023-10-18T15:59:16.363 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-18T15:59:16.899 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'rbd' created 2023-10-18T15:59:16.910 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-18T15:59:17.902 INFO:teuthology.orchestra.run.smithi082.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-18T15:59:17.914 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-18T15:59:17.914 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-18T15:59:17.915 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-18T15:59:17.916 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-18T15:59:17.917 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-18T15:59:18.248 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:59:18.258 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":4,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":86924},{"type":"v1","addr":"172.21.15.82:6825","nonce":86924}]},"active_addr":"172.21.15.82:6825/86924","active_change":"2023-10-18T15:59:12.827122+0000","active_mgr_features":4540138314316775423,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":635540307}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":665361048}]}]} 2023-10-18T15:59:18.262 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-18T15:59:18.262 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-18T15:59:18.263 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T15:59:18.552 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:59:18.553 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":10,"fsid":"cb63dacd-a4a9-467a-b30e-f9493a2182c3","created":"2023-10-18T15:59:07.909789+0000","modified":"2023-10-18T15:59:17.888808+0000","last_up_change":"2023-10-18T15:59:15.884655+0000","last_in_change":"2023-10-18T15:59:08.812513+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-10-18T15:59:15.923203+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T15:59:16.676775+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"f646c787-fe92-4e2d-abce-6b81d0e6e284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":87291},{"type":"v1","addr":"172.21.15.82:6817","nonce":87291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":87291},{"type":"v1","addr":"172.21.15.82:6819","nonce":87291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":87291},{"type":"v1","addr":"172.21.15.82:6823","nonce":87291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":87291},{"type":"v1","addr":"172.21.15.82:6821","nonce":87291}]},"public_addr":"172.21.15.82:6817/87291","cluster_addr":"172.21.15.82:6819/87291","heartbeat_back_addr":"172.21.15.82:6823/87291","heartbeat_front_addr":"172.21.15.82:6821/87291","state":["exists","up"]},{"osd":1,"uuid":"46810abb-ff37-4aeb-b272-7610274b1957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":87261},{"type":"v1","addr":"172.21.15.82:6809","nonce":87261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":87261},{"type":"v1","addr":"172.21.15.82:6811","nonce":87261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":87261},{"type":"v1","addr":"172.21.15.82:6815","nonce":87261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":87261},{"type":"v1","addr":"172.21.15.82:6813","nonce":87261}]},"public_addr":"172.21.15.82:6809/87261","cluster_addr":"172.21.15.82:6811/87261","heartbeat_back_addr":"172.21.15.82:6815/87261","heartbeat_front_addr":"172.21.15.82:6813/87261","state":["exists","up"]},{"osd":2,"uuid":"a5410593-60ce-4cb3-901e-e96645dc2c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":87262},{"type":"v1","addr":"172.21.15.82:6801","nonce":87262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":87262},{"type":"v1","addr":"172.21.15.82:6803","nonce":87262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":87262},{"type":"v1","addr":"172.21.15.82:6807","nonce":87262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":87262},{"type":"v1","addr":"172.21.15.82:6805","nonce":87262}]},"public_addr":"172.21.15.82:6801/87262","cluster_addr":"172.21.15.82:6803/87262","heartbeat_back_addr":"172.21.15.82:6807/87262","heartbeat_front_addr":"172.21.15.82:6805/87262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T15:59:14.031532+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T15:59:13.937892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T15:59:13.949251+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T15:59:18.566 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-18T15:59:18.567 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T15:59:18.850 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:59:18.851 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":10,"fsid":"cb63dacd-a4a9-467a-b30e-f9493a2182c3","created":"2023-10-18T15:59:07.909789+0000","modified":"2023-10-18T15:59:17.888808+0000","last_up_change":"2023-10-18T15:59:15.884655+0000","last_in_change":"2023-10-18T15:59:08.812513+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-10-18T15:59:15.923203+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T15:59:16.676775+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"f646c787-fe92-4e2d-abce-6b81d0e6e284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":87291},{"type":"v1","addr":"172.21.15.82:6817","nonce":87291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":87291},{"type":"v1","addr":"172.21.15.82:6819","nonce":87291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":87291},{"type":"v1","addr":"172.21.15.82:6823","nonce":87291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":87291},{"type":"v1","addr":"172.21.15.82:6821","nonce":87291}]},"public_addr":"172.21.15.82:6817/87291","cluster_addr":"172.21.15.82:6819/87291","heartbeat_back_addr":"172.21.15.82:6823/87291","heartbeat_front_addr":"172.21.15.82:6821/87291","state":["exists","up"]},{"osd":1,"uuid":"46810abb-ff37-4aeb-b272-7610274b1957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":87261},{"type":"v1","addr":"172.21.15.82:6809","nonce":87261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":87261},{"type":"v1","addr":"172.21.15.82:6811","nonce":87261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":87261},{"type":"v1","addr":"172.21.15.82:6815","nonce":87261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":87261},{"type":"v1","addr":"172.21.15.82:6813","nonce":87261}]},"public_addr":"172.21.15.82:6809/87261","cluster_addr":"172.21.15.82:6811/87261","heartbeat_back_addr":"172.21.15.82:6815/87261","heartbeat_front_addr":"172.21.15.82:6813/87261","state":["exists","up"]},{"osd":2,"uuid":"a5410593-60ce-4cb3-901e-e96645dc2c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":87262},{"type":"v1","addr":"172.21.15.82:6801","nonce":87262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":87262},{"type":"v1","addr":"172.21.15.82:6803","nonce":87262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":87262},{"type":"v1","addr":"172.21.15.82:6807","nonce":87262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":87262},{"type":"v1","addr":"172.21.15.82:6805","nonce":87262}]},"public_addr":"172.21.15.82:6801/87262","cluster_addr":"172.21.15.82:6803/87262","heartbeat_back_addr":"172.21.15.82:6807/87262","heartbeat_front_addr":"172.21.15.82:6805/87262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T15:59:14.031532+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T15:59:13.937892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T15:59:13.949251+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T15:59:18.864 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-18T15:59:19.010 INFO:teuthology.orchestra.run.smithi082.stdout:34359738370 2023-10-18T15:59:19.011 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-18T15:59:19.155 INFO:teuthology.orchestra.run.smithi082.stdout:34359738370 2023-10-18T15:59:19.157 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-18T15:59:19.301 INFO:teuthology.orchestra.run.smithi082.stdout:34359738370 2023-10-18T15:59:19.302 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-18T15:59:19.586 INFO:teuthology.orchestra.run.smithi082.stdout:34359738369 2023-10-18T15:59:19.599 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738370 got 34359738369 for osd.0 2023-10-18T15:59:20.600 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-18T15:59:20.890 INFO:teuthology.orchestra.run.smithi082.stdout:34359738369 2023-10-18T15:59:20.904 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738370 got 34359738369 for osd.0 2023-10-18T15:59:21.905 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-18T15:59:22.194 INFO:teuthology.orchestra.run.smithi082.stdout:34359738370 2023-10-18T15:59:22.208 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738370 got 34359738370 for osd.0 2023-10-18T15:59:22.208 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-18T15:59:22.495 INFO:teuthology.orchestra.run.smithi082.stdout:34359738370 2023-10-18T15:59:22.508 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738370 got 34359738370 for osd.1 2023-10-18T15:59:22.509 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-18T15:59:22.793 INFO:teuthology.orchestra.run.smithi082.stdout:34359738370 2023-10-18T15:59:22.807 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738370 got 34359738370 for osd.2 2023-10-18T15:59:22.807 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-18T15:59:22.808 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T15:59:23.093 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:59:23.095 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T15:59:23.106 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-10-18T15:59:22.832194+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":891576,"kb_used_data":288,"kb_used_omap":0,"kb_used_meta":891264,"kb_avail":282223944,"statfs":{"total":289910292480,"available":288997318656,"internally_reserved":0,"allocated":294912,"data_stored":80424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":912654336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":10,"state":"active+clean","last_fresh":"2023-10-18T15:59:17.902233+0000","last_change":"2023-10-18T15:59:17.901299+0000","last_active":"2023-10-18T15:59:17.902233+0000","last_peered":"2023-10-18T15:59:17.902233+0000","last_clean":"2023-10-18T15:59:17.902233+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T15:59:17.902233+0000","last_undegraded":"2023-10-18T15:59:17.902233+0000","last_fullsized":"2023-10-18T15:59:17.902233+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":10,"state":"active+clean","last_fresh":"2023-10-18T15:59:17.900860+0000","last_change":"2023-10-18T15:59:17.899764+0000","last_active":"2023-10-18T15:59:17.900860+0000","last_peered":"2023-10-18T15:59:17.900860+0000","last_clean":"2023-10-18T15:59:17.900860+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T15:59:17.900860+0000","last_undegraded":"2023-10-18T15:59:17.900860+0000","last_fullsized":"2023-10-18T15:59:17.900860+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":10,"state":"active+clean","last_fresh":"2023-10-18T15:59:17.900910+0000","last_change":"2023-10-18T15:59:17.899647+0000","last_active":"2023-10-18T15:59:17.900910+0000","last_peered":"2023-10-18T15:59:17.900910+0000","last_clean":"2023-10-18T15:59:17.900910+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T15:59:17.900910+0000","last_undegraded":"2023-10-18T15:59:17.900910+0000","last_fullsized":"2023-10-18T15:59:17.900910+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":10,"state":"active+clean","last_fresh":"2023-10-18T15:59:17.901299+0000","last_change":"2023-10-18T15:59:17.899072+0000","last_active":"2023-10-18T15:59:17.901299+0000","last_peered":"2023-10-18T15:59:17.901299+0000","last_clean":"2023-10-18T15:59:17.901299+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T15:59:17.901299+0000","last_undegraded":"2023-10-18T15:59:17.901299+0000","last_fullsized":"2023-10-18T15:59:17.901299+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":9,"reported_epoch":10,"state":"active+clean","last_fresh":"2023-10-18T15:59:17.901613+0000","last_change":"2023-10-18T15:59:17.900205+0000","last_active":"2023-10-18T15:59:17.901613+0000","last_peered":"2023-10-18T15:59:17.901613+0000","last_clean":"2023-10-18T15:59:17.901613+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T15:59:17.901613+0000","last_undegraded":"2023-10-18T15:59:17.901613+0000","last_fullsized":"2023-10-18T15:59:17.901613+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":10,"state":"active+clean","last_fresh":"2023-10-18T15:59:17.899025+0000","last_change":"2023-10-18T15:59:17.897709+0000","last_active":"2023-10-18T15:59:17.899025+0000","last_peered":"2023-10-18T15:59:17.899025+0000","last_clean":"2023-10-18T15:59:17.899025+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T15:59:17.899025+0000","last_undegraded":"2023-10-18T15:59:17.899025+0000","last_fullsized":"2023-10-18T15:59:17.899025+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":10,"state":"active+clean","last_fresh":"2023-10-18T15:59:17.901275+0000","last_change":"2023-10-18T15:59:17.898847+0000","last_active":"2023-10-18T15:59:17.901275+0000","last_peered":"2023-10-18T15:59:17.901275+0000","last_clean":"2023-10-18T15:59:17.901275+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T15:59:17.901275+0000","last_undegraded":"2023-10-18T15:59:17.901275+0000","last_fullsized":"2023-10-18T15:59:17.901275+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":10,"state":"active+clean","last_fresh":"2023-10-18T15:59:17.900118+0000","last_change":"2023-10-18T15:59:17.898180+0000","last_active":"2023-10-18T15:59:17.900118+0000","last_peered":"2023-10-18T15:59:17.900118+0000","last_clean":"2023-10-18T15:59:17.900118+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T15:59:17.900118+0000","last_undegraded":"2023-10-18T15:59:17.900118+0000","last_fullsized":"2023-10-18T15:59:17.900118+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":10,"state":"active+clean","last_fresh":"2023-10-18T15:59:17.900885+0000","last_change":"2023-10-18T15:59:17.899684+0000","last_active":"2023-10-18T15:59:17.900885+0000","last_peered":"2023-10-18T15:59:17.900885+0000","last_clean":"2023-10-18T15:59:17.900885+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T15:59:17.900885+0000","last_undegraded":"2023-10-18T15:59:17.900885+0000","last_fullsized":"2023-10-18T15:59:17.900885+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738371,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297192,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074648,"statfs":{"total":96636764160,"available":96332439552,"internally_reserved":0,"allocated":98304,"data_stored":26808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.19500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.183}]}]},{"osd":0,"up_from":8,"seq":34359738371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297192,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074648,"statfs":{"total":96636764160,"available":96332439552,"internally_reserved":0,"allocated":98304,"data_stored":26808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]}]},{"osd":1,"up_from":8,"seq":34359738371,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297192,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074648,"statfs":{"total":96636764160,"available":96332439552,"internally_reserved":0,"allocated":98304,"data_stored":26808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T15:59:23.107 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T15:59:23.386 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:59:23.388 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T15:59:23.399 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-10-18T15:59:22.832194+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":891576,"kb_used_data":288,"kb_used_omap":0,"kb_used_meta":891264,"kb_avail":282223944,"statfs":{"total":289910292480,"available":288997318656,"internally_reserved":0,"allocated":294912,"data_stored":80424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":912654336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":10,"state":"active+clean","last_fresh":"2023-10-18T15:59:17.902233+0000","last_change":"2023-10-18T15:59:17.901299+0000","last_active":"2023-10-18T15:59:17.902233+0000","last_peered":"2023-10-18T15:59:17.902233+0000","last_clean":"2023-10-18T15:59:17.902233+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T15:59:17.902233+0000","last_undegraded":"2023-10-18T15:59:17.902233+0000","last_fullsized":"2023-10-18T15:59:17.902233+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":10,"state":"active+clean","last_fresh":"2023-10-18T15:59:17.900860+0000","last_change":"2023-10-18T15:59:17.899764+0000","last_active":"2023-10-18T15:59:17.900860+0000","last_peered":"2023-10-18T15:59:17.900860+0000","last_clean":"2023-10-18T15:59:17.900860+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T15:59:17.900860+0000","last_undegraded":"2023-10-18T15:59:17.900860+0000","last_fullsized":"2023-10-18T15:59:17.900860+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":10,"state":"active+clean","last_fresh":"2023-10-18T15:59:17.900910+0000","last_change":"2023-10-18T15:59:17.899647+0000","last_active":"2023-10-18T15:59:17.900910+0000","last_peered":"2023-10-18T15:59:17.900910+0000","last_clean":"2023-10-18T15:59:17.900910+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T15:59:17.900910+0000","last_undegraded":"2023-10-18T15:59:17.900910+0000","last_fullsized":"2023-10-18T15:59:17.900910+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":10,"state":"active+clean","last_fresh":"2023-10-18T15:59:17.901299+0000","last_change":"2023-10-18T15:59:17.899072+0000","last_active":"2023-10-18T15:59:17.901299+0000","last_peered":"2023-10-18T15:59:17.901299+0000","last_clean":"2023-10-18T15:59:17.901299+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T15:59:17.901299+0000","last_undegraded":"2023-10-18T15:59:17.901299+0000","last_fullsized":"2023-10-18T15:59:17.901299+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":9,"reported_epoch":10,"state":"active+clean","last_fresh":"2023-10-18T15:59:17.901613+0000","last_change":"2023-10-18T15:59:17.900205+0000","last_active":"2023-10-18T15:59:17.901613+0000","last_peered":"2023-10-18T15:59:17.901613+0000","last_clean":"2023-10-18T15:59:17.901613+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T15:59:17.901613+0000","last_undegraded":"2023-10-18T15:59:17.901613+0000","last_fullsized":"2023-10-18T15:59:17.901613+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":10,"state":"active+clean","last_fresh":"2023-10-18T15:59:17.899025+0000","last_change":"2023-10-18T15:59:17.897709+0000","last_active":"2023-10-18T15:59:17.899025+0000","last_peered":"2023-10-18T15:59:17.899025+0000","last_clean":"2023-10-18T15:59:17.899025+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T15:59:17.899025+0000","last_undegraded":"2023-10-18T15:59:17.899025+0000","last_fullsized":"2023-10-18T15:59:17.899025+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":10,"state":"active+clean","last_fresh":"2023-10-18T15:59:17.901275+0000","last_change":"2023-10-18T15:59:17.898847+0000","last_active":"2023-10-18T15:59:17.901275+0000","last_peered":"2023-10-18T15:59:17.901275+0000","last_clean":"2023-10-18T15:59:17.901275+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T15:59:17.901275+0000","last_undegraded":"2023-10-18T15:59:17.901275+0000","last_fullsized":"2023-10-18T15:59:17.901275+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":10,"state":"active+clean","last_fresh":"2023-10-18T15:59:17.900118+0000","last_change":"2023-10-18T15:59:17.898180+0000","last_active":"2023-10-18T15:59:17.900118+0000","last_peered":"2023-10-18T15:59:17.900118+0000","last_clean":"2023-10-18T15:59:17.900118+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T15:59:17.900118+0000","last_undegraded":"2023-10-18T15:59:17.900118+0000","last_fullsized":"2023-10-18T15:59:17.900118+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":10,"state":"active+clean","last_fresh":"2023-10-18T15:59:17.900885+0000","last_change":"2023-10-18T15:59:17.899684+0000","last_active":"2023-10-18T15:59:17.900885+0000","last_peered":"2023-10-18T15:59:17.900885+0000","last_clean":"2023-10-18T15:59:17.900885+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T15:59:17.900885+0000","last_undegraded":"2023-10-18T15:59:17.900885+0000","last_fullsized":"2023-10-18T15:59:17.900885+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738371,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297192,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074648,"statfs":{"total":96636764160,"available":96332439552,"internally_reserved":0,"allocated":98304,"data_stored":26808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.19500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.183}]}]},{"osd":0,"up_from":8,"seq":34359738371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297192,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074648,"statfs":{"total":96636764160,"available":96332439552,"internally_reserved":0,"allocated":98304,"data_stored":26808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]}]},{"osd":1,"up_from":8,"seq":34359738371,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297192,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074648,"statfs":{"total":96636764160,"available":96332439552,"internally_reserved":0,"allocated":98304,"data_stored":26808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T15:59:23.400 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-18T15:59:23.401 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-18T15:59:23.401 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-18T15:59:23.401 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T15:59:23.733 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T15:59:23.747 INFO:teuthology.orchestra.run.smithi082.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-18T15:59:23.747 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-18T15:59:23.748 INFO:teuthology.run_tasks:Running task exec... 2023-10-18T15:59:23.773 INFO:teuthology.task.exec:Executing custom commands... 2023-10-18T15:59:23.773 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi082.front.sepia.ceph.com 2023-10-18T15:59:23.774 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool create cache 4' 2023-10-18T15:59:24.949 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cache' created 2023-10-18T15:59:24.962 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier add rbd cache' 2023-10-18T15:59:25.955 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cache' is now (or already was) a tier of 'rbd' 2023-10-18T15:59:25.968 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier cache-mode cache writeback' 2023-10-18T15:59:26.956 INFO:teuthology.orchestra.run.smithi082.stderr:set cache-mode for pool 'cache' to writeback 2023-10-18T15:59:26.968 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier set-overlay rbd cache' 2023-10-18T15:59:27.957 INFO:teuthology.orchestra.run.smithi082.stderr:overlay for 'rbd' is now (or already was) 'cache' 2023-10-18T15:59:27.969 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_type bloom' 2023-10-18T15:59:28.960 INFO:teuthology.orchestra.run.smithi082.stderr:set pool 3 hit_set_type to bloom 2023-10-18T15:59:28.973 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_count 8' 2023-10-18T15:59:29.962 INFO:teuthology.orchestra.run.smithi082.stderr:set pool 3 hit_set_count to 8 2023-10-18T15:59:29.975 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_period 60' 2023-10-18T15:59:30.966 INFO:teuthology.orchestra.run.smithi082.stderr:set pool 3 hit_set_period to 60 2023-10-18T15:59:30.978 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache target_max_objects 250' 2023-10-18T15:59:31.972 INFO:teuthology.orchestra.run.smithi082.stderr:set pool 3 target_max_objects to 250 2023-10-18T15:59:31.984 INFO:teuthology.run_tasks:Running task workunit... 2023-10-18T15:59:32.000 INFO:tasks.workunit:Pulling workunits from ref 84698e743199450ee5f6969d22ebe6bed3a68e4a 2023-10-18T15:59:32.001 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-10-18T15:59:32.001 DEBUG:teuthology.orchestra.run.smithi082:> stat -- /home/ubuntu/cephtest/mnt.0 2023-10-18T15:59:32.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T15:59:32.017 INFO:teuthology.orchestra.run.smithi082.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-18T15:59:32.018 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -- /home/ubuntu/cephtest/mnt.0 2023-10-18T15:59:32.073 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2023-10-18T15:59:32.073 DEBUG:teuthology.orchestra.run.smithi082:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2023-10-18T15:59:32.127 INFO:tasks.workunit:timeout=3h 2023-10-18T15:59:32.127 INFO:tasks.workunit:cleanup=True 2023-10-18T15:59:32.128 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 84698e743199450ee5f6969d22ebe6bed3a68e4a 2023-10-18T15:59:32.183 INFO:tasks.workunit.client.0.smithi082.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-10-18T16:00:18.999 INFO:tasks.workunit.client.0.smithi082.stderr:Note: switching to '84698e743199450ee5f6969d22ebe6bed3a68e4a'. 2023-10-18T16:00:19.000 INFO:tasks.workunit.client.0.smithi082.stderr: 2023-10-18T16:00:19.000 INFO:tasks.workunit.client.0.smithi082.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-10-18T16:00:19.000 INFO:tasks.workunit.client.0.smithi082.stderr:changes and commit them, and you can discard any commits you make in this 2023-10-18T16:00:19.000 INFO:tasks.workunit.client.0.smithi082.stderr:state without impacting any branches by switching back to a branch. 2023-10-18T16:00:19.001 INFO:tasks.workunit.client.0.smithi082.stderr: 2023-10-18T16:00:19.001 INFO:tasks.workunit.client.0.smithi082.stderr:If you want to create a new branch to retain commits you create, you may 2023-10-18T16:00:19.001 INFO:tasks.workunit.client.0.smithi082.stderr:do so (now or later) by using -c with the switch command. Example: 2023-10-18T16:00:19.001 INFO:tasks.workunit.client.0.smithi082.stderr: 2023-10-18T16:00:19.001 INFO:tasks.workunit.client.0.smithi082.stderr: git switch -c 2023-10-18T16:00:19.002 INFO:tasks.workunit.client.0.smithi082.stderr: 2023-10-18T16:00:19.002 INFO:tasks.workunit.client.0.smithi082.stderr:Or undo this operation with: 2023-10-18T16:00:19.002 INFO:tasks.workunit.client.0.smithi082.stderr: 2023-10-18T16:00:19.002 INFO:tasks.workunit.client.0.smithi082.stderr: git switch - 2023-10-18T16:00:19.002 INFO:tasks.workunit.client.0.smithi082.stderr: 2023-10-18T16:00:19.003 INFO:tasks.workunit.client.0.smithi082.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-10-18T16:00:19.003 INFO:tasks.workunit.client.0.smithi082.stderr: 2023-10-18T16:00:19.003 INFO:tasks.workunit.client.0.smithi082.stderr:HEAD is now at 84698e74319 Merge branch 'wip-62945-pacific' of https://github.com/cfsnyder/ceph into wip-yuri2-testing-2023-10-11-0853-pacific 2023-10-18T16:00:19.004 DEBUG:teuthology.orchestra.run.smithi082:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-10-18T16:00:19.089 INFO:tasks.workunit.client.0.smithi082.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-10-18T16:00:19.091 INFO:tasks.workunit.client.0.smithi082.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-18T16:00:19.091 INFO:tasks.workunit.client.0.smithi082.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-10-18T16:00:19.632 INFO:tasks.workunit.client.0.smithi082.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-10-18T16:00:19.696 INFO:tasks.workunit.client.0.smithi082.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-10-18T16:00:19.733 INFO:tasks.workunit.client.0.smithi082.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-18T16:00:19.735 INFO:tasks.workunit.client.0.smithi082.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-18T16:00:19.735 INFO:tasks.workunit.client.0.smithi082.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-10-18T16:00:19.770 INFO:tasks.workunit.client.0.smithi082.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-18T16:00:19.774 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-10-18T16:00:19.774 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-10-18T16:00:19.789 INFO:tasks.workunit:Running workunits matching rbd/cli_generic.sh on client.0... 2023-10-18T16:00:19.791 INFO:tasks.workunit:Running workunit rbd/cli_generic.sh... 2023-10-18T16:00:19.791 DEBUG:teuthology.orchestra.run.smithi082:workunit test rbd/cli_generic.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=84698e743199450ee5f6969d22ebe6bed3a68e4a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/cli_generic.sh 2023-10-18T16:00:19.849 INFO:tasks.workunit.client.0.smithi082.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/cli_generic.sh 2023-10-18T16:00:19.851 INFO:tasks.workunit.client.0.smithi082.stderr:+ . /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/../../standalone/ceph-helpers.sh 2023-10-18T16:00:19.851 INFO:tasks.workunit.client.0.smithi082.stderr:++ TIMEOUT=300 2023-10-18T16:00:19.851 INFO:tasks.workunit.client.0.smithi082.stderr:++ WAIT_FOR_CLEAN_TIMEOUT=90 2023-10-18T16:00:19.852 INFO:tasks.workunit.client.0.smithi082.stderr:++ MAX_TIMEOUT=15 2023-10-18T16:00:19.852 INFO:tasks.workunit.client.0.smithi082.stderr:++ PG_NUM=4 2023-10-18T16:00:19.852 INFO:tasks.workunit.client.0.smithi082.stderr:++ TMPDIR=/tmp 2023-10-18T16:00:19.852 INFO:tasks.workunit.client.0.smithi082.stderr:++ CEPH_BUILD_VIRTUALENV=/tmp 2023-10-18T16:00:19.852 INFO:tasks.workunit.client.0.smithi082.stderr:++ TESTDIR=/home/ubuntu/cephtest 2023-10-18T16:00:19.852 INFO:tasks.workunit.client.0.smithi082.stderr:++ type xmlstarlet 2023-10-18T16:00:19.853 INFO:tasks.workunit.client.0.smithi082.stderr:++ XMLSTARLET=xmlstarlet 2023-10-18T16:00:19.853 INFO:tasks.workunit.client.0.smithi082.stderr:+++ uname 2023-10-18T16:00:19.853 INFO:tasks.workunit.client.0.smithi082.stderr:++ '[' Linux = FreeBSD ']' 2023-10-18T16:00:19.853 INFO:tasks.workunit.client.0.smithi082.stderr:++ SED=sed 2023-10-18T16:00:19.853 INFO:tasks.workunit.client.0.smithi082.stderr:++ AWK=awk 2023-10-18T16:00:19.855 INFO:tasks.workunit.client.0.smithi082.stderr:+++ stty -a 2023-10-18T16:00:19.855 INFO:tasks.workunit.client.0.smithi082.stderr:+++ head -1 2023-10-18T16:00:19.855 INFO:tasks.workunit.client.0.smithi082.stderr:+++ sed -e 's/.*columns \([0-9]*\).*/\1/' 2023-10-18T16:00:19.855 INFO:tasks.workunit.client.0.smithi082.stderr:stty: 'standard input': Inappropriate ioctl for device 2023-10-18T16:00:19.856 INFO:tasks.workunit.client.0.smithi082.stderr:++ termwidth= 2023-10-18T16:00:19.856 INFO:tasks.workunit.client.0.smithi082.stderr:++ '[' -n '' -a '' '!=' 0 ']' 2023-10-18T16:00:19.856 INFO:tasks.workunit.client.0.smithi082.stderr:++ DIFFCOLOPTS='-y ' 2023-10-18T16:00:19.856 INFO:tasks.workunit.client.0.smithi082.stderr:++ KERNCORE=kernel.core_pattern 2023-10-18T16:00:19.856 INFO:tasks.workunit.client.0.smithi082.stderr:++ EXTRA_OPTS= 2023-10-18T16:00:19.858 INFO:tasks.workunit.client.0.smithi082.stderr:++ test '' = TESTS 2023-10-18T16:00:19.858 INFO:tasks.workunit.client.0.smithi082.stderr:+ export RBD_FORCE_ALLOW_V1=1 2023-10-18T16:00:19.859 INFO:tasks.workunit.client.0.smithi082.stderr:+ RBD_FORCE_ALLOW_V1=1 2023-10-18T16:00:19.859 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:00:19.859 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:00:19.859 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -v '^0$' 2023-10-18T16:00:19.896 INFO:tasks.workunit.client.0.smithi082.stderr:+ IMGS='testimg1 testimg2 testimg3 testimg4 testimg5 testimg6 testimg-diff1 testimg-diff2 testimg-diff3 foo foo2 bar bar2 test1 test2 test3 test4 clone2' 2023-10-18T16:00:19.896 INFO:tasks.workunit.client.0.smithi082.stderr:+ tiered=0 2023-10-18T16:00:19.897 INFO:tasks.workunit.client.0.smithi082.stderr:+ ceph osd dump 2023-10-18T16:00:19.897 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^pool' 2023-10-18T16:00:19.897 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep ''\''rbd'\''' 2023-10-18T16:00:19.897 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep tier 2023-10-18T16:00:20.275 INFO:tasks.workunit.client.0.smithi082.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 15 lfor 15/15/15 flags hashpspool tiers 3 read_tier 3 write_tier 3 stripe_width 0 application rbd 2023-10-18T16:00:20.275 INFO:tasks.workunit.client.0.smithi082.stderr:+ tiered=1 2023-10-18T16:00:20.281 INFO:tasks.workunit.client.0.smithi082.stderr:+ test_pool_image_args 2023-10-18T16:00:20.281 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'testing pool and image args...' 2023-10-18T16:00:20.281 INFO:tasks.workunit.client.0.smithi082.stdout:testing pool and image args... 2023-10-18T16:00:20.282 INFO:tasks.workunit.client.0.smithi082.stderr:+ remove_images 2023-10-18T16:00:20.282 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:20.358 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:20.424 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:20.490 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:20.556 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:20.623 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:20.698 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:20.768 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:20.835 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:20.902 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:20.968 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:21.046 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:21.111 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:21.184 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:21.251 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:21.318 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:21.592 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:21.664 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:21.731 INFO:tasks.workunit.client.0.smithi082.stderr:+ ceph osd pool delete test test --yes-i-really-really-mean-it 2023-10-18T16:00:22.111 INFO:tasks.workunit.client.0.smithi082.stderr:pool 'test' does not exist 2023-10-18T16:00:22.120 INFO:tasks.workunit.client.0.smithi082.stderr:+ ceph osd pool create test 32 2023-10-18T16:00:23.129 INFO:tasks.workunit.client.0.smithi082.stderr:pool 'test' already exists 2023-10-18T16:00:23.138 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd pool init test 2023-10-18T16:00:26.040 INFO:tasks.workunit.client.0.smithi082.stderr:+ truncate -s 1 /tmp/empty /tmp/empty@snap 2023-10-18T16:00:26.041 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:00:26.042 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:00:26.042 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 0 2023-10-18T16:00:26.073 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:00:26.073 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create -s 1 test1 2023-10-18T16:00:26.101 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:26.097+0000 7f6e98130380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:28.059 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:00:28.060 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -q test1 2023-10-18T16:00:28.087 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd import --image test2 /tmp/empty 2023-10-18T16:00:28.115 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:28.111+0000 7f7abaf66380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:28.125 INFO:tasks.workunit.client.0.smithi082.stderr: Importing image: 100% complete...done. 2023-10-18T16:00:28.126 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: --image is deprecated, use --dest 2023-10-18T16:00:28.128 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:00:28.128 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -q test2 2023-10-18T16:00:28.153 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd --dest test3 import /tmp/empty 2023-10-18T16:00:28.180 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:28.176+0000 7f6d88abb380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:28.192 INFO:tasks.workunit.client.0.smithi082.stderr: Importing image: 100% complete...done. 2023-10-18T16:00:28.195 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:00:28.195 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -q test3 2023-10-18T16:00:28.221 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd import /tmp/empty foo 2023-10-18T16:00:28.246 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:28.242+0000 7f69e31dc380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:28.256 INFO:tasks.workunit.client.0.smithi082.stderr: Importing image: 100% complete...done. 2023-10-18T16:00:28.258 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:00:28.259 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -q foo 2023-10-18T16:00:28.284 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd import --dest test/empty@snap /tmp/empty 2023-10-18T16:00:28.304 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: destination snapshot name specified for a command that doesn't use it 2023-10-18T16:00:28.305 INFO:tasks.workunit.client.0.smithi082.stderr:+ true 2023-10-18T16:00:28.306 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd import /tmp/empty test/empty@snap 2023-10-18T16:00:28.324 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: destination snapshot name specified for a command that doesn't use it 2023-10-18T16:00:28.326 INFO:tasks.workunit.client.0.smithi082.stderr:+ true 2023-10-18T16:00:28.326 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd import --image test/empty@snap /tmp/empty 2023-10-18T16:00:28.346 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: destination snapshot name specified for a command that doesn't use it 2023-10-18T16:00:28.347 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: --image is deprecated, use --dest 2023-10-18T16:00:28.347 INFO:tasks.workunit.client.0.smithi082.stderr:+ true 2023-10-18T16:00:28.347 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd import /tmp/empty@snap 2023-10-18T16:00:28.366 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: destination snapshot name specified for a command that doesn't use it 2023-10-18T16:00:28.367 INFO:tasks.workunit.client.0.smithi082.stderr:+ true 2023-10-18T16:00:28.368 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls test 2023-10-18T16:00:28.368 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:00:28.368 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 0 2023-10-18T16:00:28.393 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:00:28.393 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd import /tmp/empty test/test1 2023-10-18T16:00:28.418 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:28.414+0000 7f3dd66e8380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:30.063 INFO:tasks.workunit.client.0.smithi082.stderr: Importing image: 100% complete...done. 2023-10-18T16:00:30.066 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls test 2023-10-18T16:00:30.066 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -q test1 2023-10-18T16:00:30.095 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd -p test import /tmp/empty test2 2023-10-18T16:00:30.123 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:30.119+0000 7fb95bb31380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:30.136 INFO:tasks.workunit.client.0.smithi082.stderr: Importing image: 100% complete...done. 2023-10-18T16:00:30.136 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: -p [ --pool ] is deprecated, use --dest-pool 2023-10-18T16:00:30.138 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls test 2023-10-18T16:00:30.138 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -q test2 2023-10-18T16:00:30.165 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd --image test3 -p test import /tmp/empty 2023-10-18T16:00:30.193 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:30.189+0000 7fcef4d09380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:30.203 INFO:tasks.workunit.client.0.smithi082.stderr: Importing image: 100% complete...done. 2023-10-18T16:00:30.204 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: --image is deprecated, use --dest 2023-10-18T16:00:30.204 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: -p [ --pool ] is deprecated, use --dest-pool 2023-10-18T16:00:30.206 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls test 2023-10-18T16:00:30.206 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -q test3 2023-10-18T16:00:30.232 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd --image test4 -p test import /tmp/empty 2023-10-18T16:00:30.260 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:30.256+0000 7f9d9e69a380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:30.274 INFO:tasks.workunit.client.0.smithi082.stderr: Importing image: 100% complete...done. 2023-10-18T16:00:30.275 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: --image is deprecated, use --dest 2023-10-18T16:00:30.275 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: -p [ --pool ] is deprecated, use --dest-pool 2023-10-18T16:00:30.277 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls test 2023-10-18T16:00:30.277 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -q test4 2023-10-18T16:00:30.303 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd --dest test5 -p test import /tmp/empty 2023-10-18T16:00:30.331 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:30.327+0000 7f4f16a88380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:30.339 INFO:tasks.workunit.client.0.smithi082.stderr: Importing image: 100% complete...done. 2023-10-18T16:00:30.340 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: -p [ --pool ] is deprecated, use --dest-pool 2023-10-18T16:00:30.342 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls test 2023-10-18T16:00:30.342 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -q test5 2023-10-18T16:00:30.368 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd --dest test6 --dest-pool test import /tmp/empty 2023-10-18T16:00:30.395 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:30.391+0000 7f3a0bba9380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:30.407 INFO:tasks.workunit.client.0.smithi082.stderr: Importing image: 100% complete...done. 2023-10-18T16:00:30.409 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls test 2023-10-18T16:00:30.410 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -q test6 2023-10-18T16:00:30.436 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd --image test7 --dest-pool test import /tmp/empty 2023-10-18T16:00:30.464 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:30.460+0000 7efcc112e380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:30.472 INFO:tasks.workunit.client.0.smithi082.stderr: Importing image: 100% complete...done. 2023-10-18T16:00:30.473 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: --image is deprecated, use --dest 2023-10-18T16:00:30.475 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls test 2023-10-18T16:00:30.475 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -q test7 2023-10-18T16:00:30.502 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd --image test/test8 import /tmp/empty 2023-10-18T16:00:30.527 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:30.523+0000 7f07e7a8c380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:30.536 INFO:tasks.workunit.client.0.smithi082.stderr: Importing image: 100% complete...done. 2023-10-18T16:00:30.537 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: --image is deprecated, use --dest 2023-10-18T16:00:30.539 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls test 2023-10-18T16:00:30.539 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -q test8 2023-10-18T16:00:30.565 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd --dest test/test9 import /tmp/empty 2023-10-18T16:00:30.591 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:30.587+0000 7fa8203aa380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:30.602 INFO:tasks.workunit.client.0.smithi082.stderr: Importing image: 100% complete...done. 2023-10-18T16:00:30.605 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls test 2023-10-18T16:00:30.605 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -q test9 2023-10-18T16:00:30.631 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd import --pool test /tmp/empty 2023-10-18T16:00:30.658 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:30.654+0000 7fd71006d380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:30.668 INFO:tasks.workunit.client.0.smithi082.stderr: Importing image: 100% complete...done. 2023-10-18T16:00:30.669 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: -p [ --pool ] is deprecated, use --dest-pool 2023-10-18T16:00:30.671 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls test 2023-10-18T16:00:30.671 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -q empty 2023-10-18T16:00:30.697 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd copy test/test9 test10 2023-10-18T16:00:32.085 INFO:tasks.workunit.client.0.smithi082.stderr: Image copy: 100% complete... Image copy: 100% complete...done. 2023-10-18T16:00:32.088 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls test 2023-10-18T16:00:32.089 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -qv test10 2023-10-18T16:00:32.117 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:00:32.118 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -q test10 2023-10-18T16:00:32.144 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd copy test/test9 test/test10 2023-10-18T16:00:32.194 INFO:tasks.workunit.client.0.smithi082.stderr: Image copy: 100% complete... Image copy: 100% complete...done. 2023-10-18T16:00:32.197 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls test 2023-10-18T16:00:32.197 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -q test10 2023-10-18T16:00:32.223 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd copy --pool test test10 --dest-pool test test11 2023-10-18T16:00:32.274 INFO:tasks.workunit.client.0.smithi082.stderr: Image copy: 100% complete... Image copy: 100% complete...done. 2023-10-18T16:00:32.277 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls test 2023-10-18T16:00:32.277 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -q test11 2023-10-18T16:00:32.304 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd copy --dest-pool rbd --pool test test11 test12 2023-10-18T16:00:32.354 INFO:tasks.workunit.client.0.smithi082.stderr: Image copy: 100% complete... Image copy: 100% complete...done. 2023-10-18T16:00:32.357 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:00:32.357 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test12 2023-10-18T16:00:32.382 INFO:tasks.workunit.client.0.smithi082.stdout:test12 2023-10-18T16:00:32.383 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls test 2023-10-18T16:00:32.383 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -qv test12 2023-10-18T16:00:32.409 INFO:tasks.workunit.client.0.smithi082.stderr:+ rm -f /tmp/empty /tmp/empty@snap 2023-10-18T16:00:32.410 INFO:tasks.workunit.client.0.smithi082.stderr:+ ceph osd pool delete test test --yes-i-really-really-mean-it 2023-10-18T16:00:33.173 INFO:tasks.workunit.client.0.smithi082.stderr:pool 'test' does not exist 2023-10-18T16:00:33.183 INFO:tasks.workunit.client.0.smithi082.stderr:+ for f in foo test1 test10 test12 test2 test3 2023-10-18T16:00:33.183 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm foo 2023-10-18T16:00:33.228 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:33.231 INFO:tasks.workunit.client.0.smithi082.stderr:+ for f in foo test1 test10 test12 test2 test3 2023-10-18T16:00:33.231 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test1 2023-10-18T16:00:33.266 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:33.268 INFO:tasks.workunit.client.0.smithi082.stderr:+ for f in foo test1 test10 test12 test2 test3 2023-10-18T16:00:33.269 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test10 2023-10-18T16:00:33.343 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:33.346 INFO:tasks.workunit.client.0.smithi082.stderr:+ for f in foo test1 test10 test12 test2 test3 2023-10-18T16:00:33.346 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test12 2023-10-18T16:00:33.418 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:33.420 INFO:tasks.workunit.client.0.smithi082.stderr:+ for f in foo test1 test10 test12 test2 test3 2023-10-18T16:00:33.420 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test2 2023-10-18T16:00:33.462 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:33.465 INFO:tasks.workunit.client.0.smithi082.stderr:+ for f in foo test1 test10 test12 test2 test3 2023-10-18T16:00:33.465 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test3 2023-10-18T16:00:33.499 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:33.501 INFO:tasks.workunit.client.0.smithi082.stderr:+ test_rename 2023-10-18T16:00:33.502 INFO:tasks.workunit.client.0.smithi082.stdout:testing rename... 2023-10-18T16:00:33.502 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'testing rename...' 2023-10-18T16:00:33.503 INFO:tasks.workunit.client.0.smithi082.stderr:+ remove_images 2023-10-18T16:00:33.503 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:33.569 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:33.635 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:33.701 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:33.768 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:33.834 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:33.900 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:33.968 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:34.034 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:34.109 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:34.184 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:34.257 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:34.330 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:34.403 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:34.469 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:34.535 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:34.602 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:34.669 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:34.736 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 1 -s 1 foo 2023-10-18T16:00:34.758 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: image format 1 is deprecated 2023-10-18T16:00:34.764 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:34.760+0000 7ff21b27e380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:34.769 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 -s 1 bar 2023-10-18T16:00:34.813 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rename foo foo2 2023-10-18T16:00:34.862 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rename foo2 bar 2023-10-18T16:00:34.863 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep exists 2023-10-18T16:00:34.897 INFO:tasks.workunit.client.0.smithi082.stdout:2023-10-18T16:00:34.888+0000 7fd1da13d380 -1 librbd::Operations: rbd image bar already exists 2023-10-18T16:00:34.897 INFO:tasks.workunit.client.0.smithi082.stdout:rbd: rename error: (17) File exists 2023-10-18T16:00:34.897 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rename bar bar2 2023-10-18T16:00:34.957 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rename bar2 foo2 2023-10-18T16:00:34.957 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep exists 2023-10-18T16:00:34.994 INFO:tasks.workunit.client.0.smithi082.stdout:2023-10-18T16:00:34.985+0000 7f3488555380 -1 librbd::Operations: rbd image foo2 already exists 2023-10-18T16:00:34.994 INFO:tasks.workunit.client.0.smithi082.stdout:rbd: rename error: (17) File exists 2023-10-18T16:00:34.995 INFO:tasks.workunit.client.0.smithi082.stderr:+ ceph osd pool create rbd2 8 2023-10-18T16:00:36.180 INFO:tasks.workunit.client.0.smithi082.stderr:pool 'rbd2' already exists 2023-10-18T16:00:36.189 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd pool init rbd2 2023-10-18T16:00:38.894 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create -p rbd2 -s 1 foo 2023-10-18T16:00:38.927 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:38.923+0000 7f36eeb28380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:40.902 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rename rbd2/foo rbd2/bar 2023-10-18T16:00:40.949 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd -p rbd2 ls 2023-10-18T16:00:40.950 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep bar 2023-10-18T16:00:40.978 INFO:tasks.workunit.client.0.smithi082.stdout:bar 2023-10-18T16:00:40.978 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rename rbd2/bar foo 2023-10-18T16:00:41.023 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rename --pool rbd2 foo bar 2023-10-18T16:00:41.071 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rename rbd2/bar --dest-pool rbd foo 2023-10-18T16:00:41.092 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: mv/rename across pools not supported 2023-10-18T16:00:41.093 INFO:tasks.workunit.client.0.smithi082.stderr:source pool: rbd2 dest pool: rbd 2023-10-18T16:00:41.093 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rename --pool rbd2 bar --dest-pool rbd2 foo 2023-10-18T16:00:41.140 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd -p rbd2 ls 2023-10-18T16:00:41.140 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep foo 2023-10-18T16:00:41.167 INFO:tasks.workunit.client.0.smithi082.stdout:foo 2023-10-18T16:00:41.167 INFO:tasks.workunit.client.0.smithi082.stderr:+ ceph osd pool rm rbd2 rbd2 --yes-i-really-really-mean-it 2023-10-18T16:00:42.003 INFO:tasks.workunit.client.0.smithi082.stderr:pool 'rbd2' does not exist 2023-10-18T16:00:42.012 INFO:tasks.workunit.client.0.smithi082.stderr:+ remove_images 2023-10-18T16:00:42.013 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:42.098 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:42.172 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:42.238 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:42.305 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:42.371 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:42.438 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:42.504 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:42.571 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:42.644 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:42.919 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:42.998 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:43.069 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:43.197 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:43.264 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:43.330 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:43.397 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:43.463 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:43.530 INFO:tasks.workunit.client.0.smithi082.stderr:+ test_ls 2023-10-18T16:00:43.530 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'testing ls...' 2023-10-18T16:00:43.531 INFO:tasks.workunit.client.0.smithi082.stdout:testing ls... 2023-10-18T16:00:43.531 INFO:tasks.workunit.client.0.smithi082.stderr:+ remove_images 2023-10-18T16:00:43.532 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:43.597 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:43.663 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:43.930 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:44.003 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:44.069 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:44.135 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:44.203 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:44.272 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:44.343 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:44.408 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:44.474 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:44.542 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:44.609 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:44.675 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:44.742 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:44.808 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:44.874 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:44.942 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 1 -s 1 test1 2023-10-18T16:00:44.964 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: image format 1 is deprecated 2023-10-18T16:00:44.969 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:44.967+0000 7fa4a5df6380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:44.975 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 1 -s 1 test2 2023-10-18T16:00:44.997 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: image format 1 is deprecated 2023-10-18T16:00:45.002 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:45.000+0000 7fb72f7e7380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:45.007 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:00:45.008 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test1 2023-10-18T16:00:45.033 INFO:tasks.workunit.client.0.smithi082.stdout:test1 2023-10-18T16:00:45.034 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:00:45.034 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test2 2023-10-18T16:00:45.058 INFO:tasks.workunit.client.0.smithi082.stdout:test2 2023-10-18T16:00:45.059 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:00:45.060 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:00:45.060 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 2 2023-10-18T16:00:45.084 INFO:tasks.workunit.client.0.smithi082.stdout:2 2023-10-18T16:00:45.085 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls -l 2023-10-18T16:00:45.085 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'test1.*1 MiB.*1' 2023-10-18T16:00:45.119 INFO:tasks.workunit.client.0.smithi082.stdout:test1 1 MiB 1 2023-10-18T16:00:45.120 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls -l 2023-10-18T16:00:45.120 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'test2.*1 MiB.*1' 2023-10-18T16:00:45.151 INFO:tasks.workunit.client.0.smithi082.stdout:test2 1 MiB 1 2023-10-18T16:00:45.152 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test1 2023-10-18T16:00:45.189 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:45.192 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test2 2023-10-18T16:00:45.229 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:45.231 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 -s 1 test1 2023-10-18T16:00:45.274 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 -s 1 test2 2023-10-18T16:00:45.316 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:00:45.317 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test1 2023-10-18T16:00:45.342 INFO:tasks.workunit.client.0.smithi082.stdout:test1 2023-10-18T16:00:45.342 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:00:45.343 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test2 2023-10-18T16:00:45.368 INFO:tasks.workunit.client.0.smithi082.stdout:test2 2023-10-18T16:00:45.368 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:00:45.369 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:00:45.369 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 2 2023-10-18T16:00:45.394 INFO:tasks.workunit.client.0.smithi082.stdout:2 2023-10-18T16:00:45.394 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls -l 2023-10-18T16:00:45.395 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'test1.*1 MiB.*2' 2023-10-18T16:00:45.429 INFO:tasks.workunit.client.0.smithi082.stdout:test1 1 MiB 2 2023-10-18T16:00:45.429 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls -l 2023-10-18T16:00:45.430 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'test2.*1 MiB.*2' 2023-10-18T16:00:45.463 INFO:tasks.workunit.client.0.smithi082.stdout:test2 1 MiB 2 2023-10-18T16:00:45.464 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test1 2023-10-18T16:00:45.565 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:45.568 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test2 2023-10-18T16:00:45.648 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:45.650 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 -s 1 test1 2023-10-18T16:00:45.690 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 1 -s 1 test2 2023-10-18T16:00:45.712 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: image format 1 is deprecated 2023-10-18T16:00:45.717 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:45.715+0000 7fba2e16c380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:45.723 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:00:45.723 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test1 2023-10-18T16:00:45.749 INFO:tasks.workunit.client.0.smithi082.stdout:test1 2023-10-18T16:00:45.749 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:00:45.750 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test2 2023-10-18T16:00:45.775 INFO:tasks.workunit.client.0.smithi082.stdout:test2 2023-10-18T16:00:45.775 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:00:45.776 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:00:45.776 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 2 2023-10-18T16:00:45.801 INFO:tasks.workunit.client.0.smithi082.stdout:2 2023-10-18T16:00:45.801 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls -l 2023-10-18T16:00:45.802 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'test1.*1 MiB.*2' 2023-10-18T16:00:45.835 INFO:tasks.workunit.client.0.smithi082.stdout:test1 1 MiB 2 2023-10-18T16:00:45.836 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls -l 2023-10-18T16:00:45.836 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'test2.*1 MiB.*1' 2023-10-18T16:00:45.869 INFO:tasks.workunit.client.0.smithi082.stdout:test2 1 MiB 1 2023-10-18T16:00:45.870 INFO:tasks.workunit.client.0.smithi082.stderr:+ remove_images 2023-10-18T16:00:45.870 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:45.938 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:46.011 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:46.084 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:46.158 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:46.234 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:46.304 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:46.377 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:46.453 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:46.523 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:46.595 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:46.661 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:46.729 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:46.795 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:46.922 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:46.993 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:47.059 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:47.126 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:00:47.195 INFO:tasks.workunit.client.0.smithi082.stderr:++ seq -w 00 99 2023-10-18T16:00:47.204 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:47.205 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.00 -s 1 2023-10-18T16:00:47.231 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:47.228+0000 7f15bdb17380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:47.240 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:47.241 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.01 -s 1 2023-10-18T16:00:47.267 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:47.264+0000 7efd22069380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:47.272 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:47.273 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.02 -s 1 2023-10-18T16:00:47.299 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:47.296+0000 7efeb9a87380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:47.308 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:47.309 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.03 -s 1 2023-10-18T16:00:47.335 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:47.332+0000 7f8bf231c380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:47.344 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:47.344 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.04 -s 1 2023-10-18T16:00:47.370 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:47.367+0000 7fcfe7be1380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:47.379 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:47.380 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.05 -s 1 2023-10-18T16:00:47.406 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:47.403+0000 7f5f48025380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:47.411 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:47.412 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.06 -s 1 2023-10-18T16:00:47.438 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:47.435+0000 7f256e646380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:47.444 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:47.444 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.07 -s 1 2023-10-18T16:00:47.470 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:47.467+0000 7f70be351380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:47.476 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:47.476 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.08 -s 1 2023-10-18T16:00:47.503 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:47.500+0000 7f4af98e6380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:47.508 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:47.508 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.09 -s 1 2023-10-18T16:00:47.535 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:47.532+0000 7f01c22e6380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:47.540 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:47.541 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.10 -s 1 2023-10-18T16:00:47.567 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:47.564+0000 7f651b229380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:47.573 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:47.573 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.11 -s 1 2023-10-18T16:00:47.600 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:47.597+0000 7f2d9686c380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:47.605 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:47.605 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.12 -s 1 2023-10-18T16:00:47.632 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:47.629+0000 7fbb4ff33380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:47.637 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:47.637 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.13 -s 1 2023-10-18T16:00:47.664 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:47.661+0000 7fbb90ded380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:47.669 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:47.669 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.14 -s 1 2023-10-18T16:00:47.696 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:47.693+0000 7f14eb1f8380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:47.704 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:47.704 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.15 -s 1 2023-10-18T16:00:47.731 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:47.728+0000 7f9009e24380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:47.740 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:47.740 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.16 -s 1 2023-10-18T16:00:47.766 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:47.763+0000 7ff4a244e380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:47.776 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:47.776 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.17 -s 1 2023-10-18T16:00:47.802 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:47.800+0000 7f044d02f380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:47.813 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:47.814 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.18 -s 1 2023-10-18T16:00:47.840 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:47.837+0000 7fab4be28380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:47.851 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:47.852 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.19 -s 1 2023-10-18T16:00:47.878 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:47.876+0000 7f4f9c661380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:47.886 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:47.886 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.20 -s 1 2023-10-18T16:00:47.913 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:47.910+0000 7f09ecbaf380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:47.922 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:47.922 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.21 -s 1 2023-10-18T16:00:47.948 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:47.945+0000 7ff72b9e2380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:47.960 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:47.960 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.22 -s 1 2023-10-18T16:00:47.986 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:47.984+0000 7f673b9ea380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:47.995 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:47.995 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.23 -s 1 2023-10-18T16:00:48.022 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.019+0000 7f2c8c464380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.030 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.030 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.24 -s 1 2023-10-18T16:00:48.057 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.054+0000 7f3e54feb380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.064 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.064 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.25 -s 1 2023-10-18T16:00:48.091 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.088+0000 7fae5e54a380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.097 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.098 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.26 -s 1 2023-10-18T16:00:48.124 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.122+0000 7f847449c380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.130 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.130 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.27 -s 1 2023-10-18T16:00:48.157 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.154+0000 7fe7ffa91380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.163 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.163 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.28 -s 1 2023-10-18T16:00:48.190 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.187+0000 7faad2b92380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.196 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.196 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.29 -s 1 2023-10-18T16:00:48.223 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.220+0000 7ff7c92f7380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.229 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.229 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.30 -s 1 2023-10-18T16:00:48.257 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.254+0000 7f3a49930380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.269 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.269 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.31 -s 1 2023-10-18T16:00:48.296 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.293+0000 7f59799bf380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.307 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.307 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.32 -s 1 2023-10-18T16:00:48.334 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.331+0000 7f36a278f380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.339 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.340 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.33 -s 1 2023-10-18T16:00:48.366 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.363+0000 7efdd61f9380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.372 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.373 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.34 -s 1 2023-10-18T16:00:48.399 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.396+0000 7f0c5796e380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.405 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.406 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.35 -s 1 2023-10-18T16:00:48.433 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.430+0000 7f882aeba380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.438 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.438 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.36 -s 1 2023-10-18T16:00:48.464 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.462+0000 7f2a8ff62380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.470 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.470 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.37 -s 1 2023-10-18T16:00:48.497 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.494+0000 7fba40846380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.502 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.503 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.38 -s 1 2023-10-18T16:00:48.529 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.526+0000 7f486ca2a380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.540 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.540 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.39 -s 1 2023-10-18T16:00:48.567 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.564+0000 7f73f3d7c380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.578 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.578 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.40 -s 1 2023-10-18T16:00:48.605 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.602+0000 7fca558f2380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.614 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.614 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.41 -s 1 2023-10-18T16:00:48.641 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.638+0000 7f1f29100380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.648 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.648 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.42 -s 1 2023-10-18T16:00:48.675 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.672+0000 7fb3ab386380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.686 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.686 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.43 -s 1 2023-10-18T16:00:48.713 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.710+0000 7ffa4b58b380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.719 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.720 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.44 -s 1 2023-10-18T16:00:48.747 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.744+0000 7f35ce500380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.752 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.752 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.45 -s 1 2023-10-18T16:00:48.779 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.777+0000 7f5d220e4380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.789 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.789 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.46 -s 1 2023-10-18T16:00:48.816 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.813+0000 7fc85a0ca380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.824 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.825 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.47 -s 1 2023-10-18T16:00:48.851 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.849+0000 7f1bd6377380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.858 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.858 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.48 -s 1 2023-10-18T16:00:48.885 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.882+0000 7f51181f6380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.893 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.893 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.49 -s 1 2023-10-18T16:00:48.920 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.917+0000 7f3c74e3e380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.926 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.926 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.50 -s 1 2023-10-18T16:00:48.953 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.950+0000 7f0843cd0380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.961 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.961 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.51 -s 1 2023-10-18T16:00:48.988 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:48.985+0000 7fa9f4ac4380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:48.995 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:48.996 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.52 -s 1 2023-10-18T16:00:49.022 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.019+0000 7f4ea5505380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.033 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.033 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.53 -s 1 2023-10-18T16:00:49.060 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.057+0000 7fa8122df380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.069 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.069 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.54 -s 1 2023-10-18T16:00:49.096 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.093+0000 7f290b74b380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.105 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.106 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.55 -s 1 2023-10-18T16:00:49.132 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.129+0000 7f462509b380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.137 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.137 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.56 -s 1 2023-10-18T16:00:49.164 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.161+0000 7f74dc5a9380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.169 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.169 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.57 -s 1 2023-10-18T16:00:49.196 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.193+0000 7fa8d5585380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.202 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.202 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.58 -s 1 2023-10-18T16:00:49.229 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.226+0000 7f6cc1fff380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.234 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.235 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.59 -s 1 2023-10-18T16:00:49.261 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.258+0000 7f5a85a7b380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.270 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.271 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.60 -s 1 2023-10-18T16:00:49.294 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.292+0000 7f7b881ae380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.305 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.306 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.61 -s 1 2023-10-18T16:00:49.332 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.330+0000 7f62127fa380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.340 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.340 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.62 -s 1 2023-10-18T16:00:49.367 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.364+0000 7f9b850f3380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.377 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.377 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.63 -s 1 2023-10-18T16:00:49.403 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.400+0000 7fbd6a3a3380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.411 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.412 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.64 -s 1 2023-10-18T16:00:49.438 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.435+0000 7fbf6182b380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.447 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.447 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.65 -s 1 2023-10-18T16:00:49.474 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.471+0000 7f4d9a619380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.479 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.480 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.66 -s 1 2023-10-18T16:00:49.506 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.504+0000 7f03cdfa6380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.517 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.518 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.67 -s 1 2023-10-18T16:00:49.544 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.541+0000 7f9c0ec3f380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.555 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.555 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.68 -s 1 2023-10-18T16:00:49.582 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.579+0000 7f3c8a000380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.589 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.589 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.69 -s 1 2023-10-18T16:00:49.616 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.613+0000 7fde23670380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.633 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.633 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.70 -s 1 2023-10-18T16:00:49.660 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.657+0000 7f5521a65380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.670 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.671 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.71 -s 1 2023-10-18T16:00:49.697 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.694+0000 7f1b4a61e380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.704 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.704 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.72 -s 1 2023-10-18T16:00:49.731 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.728+0000 7f22c997d380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.738 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.739 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.73 -s 1 2023-10-18T16:00:49.765 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.763+0000 7f51563c9380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.773 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.773 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.74 -s 1 2023-10-18T16:00:49.799 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.797+0000 7f8042a73380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.805 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.806 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.75 -s 1 2023-10-18T16:00:49.832 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.829+0000 7f3be80cb380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.840 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.840 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.76 -s 1 2023-10-18T16:00:49.867 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.864+0000 7f3205b9e380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.873 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.873 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.77 -s 1 2023-10-18T16:00:49.899 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.896+0000 7f172d3c7380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.906 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.906 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.78 -s 1 2023-10-18T16:00:49.933 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.930+0000 7f26e6029380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.938 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.939 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.79 -s 1 2023-10-18T16:00:49.965 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.962+0000 7f5b4c009380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:49.971 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:49.971 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.80 -s 1 2023-10-18T16:00:49.998 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:49.995+0000 7f80a337d380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:50.003 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:50.004 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.81 -s 1 2023-10-18T16:00:50.030 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:50.027+0000 7f2b92dd5380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:50.038 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:50.038 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.82 -s 1 2023-10-18T16:00:50.064 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:50.062+0000 7f400377e380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:50.071 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:50.071 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.83 -s 1 2023-10-18T16:00:50.098 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:50.095+0000 7ff287ac2380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:50.104 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:50.104 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.84 -s 1 2023-10-18T16:00:50.131 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:50.127+0000 7fc212806380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:50.136 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:50.136 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.85 -s 1 2023-10-18T16:00:50.163 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:50.160+0000 7f2934dc9380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:50.169 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:50.169 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.86 -s 1 2023-10-18T16:00:50.196 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:50.193+0000 7fee7142c380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:50.204 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:50.204 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.87 -s 1 2023-10-18T16:00:50.231 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:50.228+0000 7f3b46e25380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:50.241 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:50.241 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.88 -s 1 2023-10-18T16:00:50.268 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:50.266+0000 7fb4d33e9380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:50.279 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:50.279 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.89 -s 1 2023-10-18T16:00:50.306 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:50.303+0000 7fbc4f66b380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:50.315 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:50.315 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.90 -s 1 2023-10-18T16:00:50.342 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:50.339+0000 7f75cc398380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:50.350 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:50.350 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.91 -s 1 2023-10-18T16:00:50.377 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:50.374+0000 7f975988d380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:50.388 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:50.388 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.92 -s 1 2023-10-18T16:00:50.414 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:50.411+0000 7fbc04fd2380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:50.421 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:50.421 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.93 -s 1 2023-10-18T16:00:50.448 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:50.445+0000 7fccc93b2380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:50.458 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:50.459 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.94 -s 1 2023-10-18T16:00:50.485 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:50.483+0000 7fb3bd41e380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:50.496 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:50.496 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.95 -s 1 2023-10-18T16:00:50.524 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:50.521+0000 7fa4f823f380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:50.530 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:50.530 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.96 -s 1 2023-10-18T16:00:50.558 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:50.554+0000 7fd4bff04380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:50.563 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:50.563 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.97 -s 1 2023-10-18T16:00:50.590 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:50.587+0000 7f9fc34ef380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:50.595 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:50.596 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.98 -s 1 2023-10-18T16:00:50.622 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:50.619+0000 7ffaef90c380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:50.627 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:50.627 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.99 -s 1 2023-10-18T16:00:50.654 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:50.651+0000 7fc4f9380380 -1 librbd: Forced V1 image creation. 2023-10-18T16:00:50.661 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:00:50.661 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:00:50.661 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 100 2023-10-18T16:00:50.686 INFO:tasks.workunit.client.0.smithi082.stdout:100 2023-10-18T16:00:50.688 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls -l 2023-10-18T16:00:50.688 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep image 2023-10-18T16:00:50.688 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:00:50.689 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 100 2023-10-18T16:00:50.750 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:50.747+0000 7ff562225700 0 -- 172.21.15.82:0/2204230907 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x55a62f407c20 msgr2=0x55a62f4a5d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:00:50.759 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:50.756+0000 7ff562225700 0 -- 172.21.15.82:0/2204230907 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x7ff544036350 msgr2=0x7ff544038780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:00:50.920 INFO:tasks.workunit.client.0.smithi082.stdout:100 2023-10-18T16:00:50.921 INFO:tasks.workunit.client.0.smithi082.stderr:++ seq -w 00 99 2023-10-18T16:00:50.922 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:50.922 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.00 2023-10-18T16:00:50.955 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:50.957 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:50.957 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.01 2023-10-18T16:00:50.996 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:50.998 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:50.998 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.02 2023-10-18T16:00:51.032 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:51.035 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:51.035 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.03 2023-10-18T16:00:51.073 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:51.076 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:51.076 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.04 2023-10-18T16:00:51.113 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:51.116 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:51.116 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.05 2023-10-18T16:00:51.151 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:51.153 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:51.154 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.06 2023-10-18T16:00:51.188 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:51.190 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:51.190 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.07 2023-10-18T16:00:51.225 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:51.227 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:51.227 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.08 2023-10-18T16:00:51.261 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:51.263 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:51.264 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.09 2023-10-18T16:00:51.299 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:51.301 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:51.301 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.10 2023-10-18T16:00:51.338 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:51.339 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:51.339 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.11 2023-10-18T16:00:51.575 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:51.577 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:51.577 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.12 2023-10-18T16:00:51.615 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:51.618 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:51.618 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.13 2023-10-18T16:00:51.656 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:51.659 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:51.659 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.14 2023-10-18T16:00:51.698 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:51.700 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:51.701 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.15 2023-10-18T16:00:51.741 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:51.743 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:51.744 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.16 2023-10-18T16:00:51.781 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:51.784 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:51.784 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.17 2023-10-18T16:00:51.821 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:51.824 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:51.824 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.18 2023-10-18T16:00:51.863 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:52.027 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:52.028 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.19 2023-10-18T16:00:52.028 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:52.028 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:52.028 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.20 2023-10-18T16:00:52.029 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:52.029 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:52.030 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.21 2023-10-18T16:00:52.030 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:52.030 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:52.030 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.22 2023-10-18T16:00:52.033 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:52.035 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:52.036 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.23 2023-10-18T16:00:52.071 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:52.073 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:52.073 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.24 2023-10-18T16:00:52.108 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:52.111 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:52.111 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.25 2023-10-18T16:00:52.145 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:52.148 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:52.148 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.26 2023-10-18T16:00:52.182 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:52.185 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:52.185 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.27 2023-10-18T16:00:52.222 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:52.224 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:52.224 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.28 2023-10-18T16:00:52.261 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:52.264 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:52.264 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.29 2023-10-18T16:00:52.298 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:52.301 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:52.301 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.30 2023-10-18T16:00:52.338 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:52.339 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:52.339 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.31 2023-10-18T16:00:52.373 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:52.375 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:52.375 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.32 2023-10-18T16:00:52.410 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:52.413 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:52.413 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.33 2023-10-18T16:00:52.455 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:52.457 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:52.458 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.34 2023-10-18T16:00:52.492 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:52.495 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:52.495 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.35 2023-10-18T16:00:52.531 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:52.533 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:52.533 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.36 2023-10-18T16:00:52.572 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:52.574 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:52.574 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.37 2023-10-18T16:00:52.615 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:52.617 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:52.617 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.38 2023-10-18T16:00:52.651 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:52.653 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:52.654 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.39 2023-10-18T16:00:52.691 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:52.693 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:52.694 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.40 2023-10-18T16:00:52.730 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:52.732 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:52.732 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.41 2023-10-18T16:00:52.953 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:00:52.950+0000 7ff756a5d700 0 --2- 172.21.15.82:0/3306072267 >> v2:172.21.15.82:3300/0 conn(0x55a78a72dc60 0x55a78a7c30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2023-10-18T16:00:52.970 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:52.972 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:52.972 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.42 2023-10-18T16:00:53.009 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:53.012 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:53.012 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.43 2023-10-18T16:00:53.049 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:53.051 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:53.051 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.44 2023-10-18T16:00:53.086 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:53.088 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:53.088 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.45 2023-10-18T16:00:53.125 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:53.128 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:53.128 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.46 2023-10-18T16:00:53.162 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:53.164 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:53.164 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.47 2023-10-18T16:00:53.204 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:53.206 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:53.206 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.48 2023-10-18T16:00:53.241 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:53.243 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:53.243 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.49 2023-10-18T16:00:53.278 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:53.280 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:53.280 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.50 2023-10-18T16:00:53.314 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:53.317 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:53.317 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.51 2023-10-18T16:00:53.357 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:53.360 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:53.360 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.52 2023-10-18T16:00:53.400 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:53.402 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:53.402 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.53 2023-10-18T16:00:53.438 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:53.440 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:53.441 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.54 2023-10-18T16:00:53.677 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:53.680 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:53.680 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.55 2023-10-18T16:00:53.720 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:53.723 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:53.723 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.56 2023-10-18T16:00:53.765 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:53.767 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:53.768 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.57 2023-10-18T16:00:53.805 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:53.808 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:53.808 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.58 2023-10-18T16:00:53.842 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:53.845 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:53.845 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.59 2023-10-18T16:00:53.880 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:53.883 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:53.883 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.60 2023-10-18T16:00:53.926 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:53.928 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:53.928 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.61 2023-10-18T16:00:53.963 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:53.965 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:53.965 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.62 2023-10-18T16:00:54.002 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:54.004 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:54.005 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.63 2023-10-18T16:00:54.043 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:54.046 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:54.046 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.64 2023-10-18T16:00:54.082 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:54.085 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:54.085 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.65 2023-10-18T16:00:54.122 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:54.124 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:54.124 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.66 2023-10-18T16:00:54.158 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:54.161 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:54.161 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.67 2023-10-18T16:00:54.196 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:54.199 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:54.199 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.68 2023-10-18T16:00:54.238 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:54.240 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:54.240 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.69 2023-10-18T16:00:54.279 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:54.282 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:54.282 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.70 2023-10-18T16:00:54.317 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:54.319 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:54.319 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.71 2023-10-18T16:00:54.554 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:54.557 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:54.557 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.72 2023-10-18T16:00:54.591 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:54.593 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:54.594 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.73 2023-10-18T16:00:54.634 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:54.635 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:54.636 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.74 2023-10-18T16:00:54.673 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:54.675 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:54.675 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.75 2023-10-18T16:00:54.711 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:54.713 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:54.714 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.76 2023-10-18T16:00:54.748 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:54.751 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:54.751 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.77 2023-10-18T16:00:54.786 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:54.788 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:54.789 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.78 2023-10-18T16:00:54.823 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:54.825 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:54.826 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.79 2023-10-18T16:00:54.869 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:54.872 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:54.872 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.80 2023-10-18T16:00:54.910 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:54.913 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:54.913 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.81 2023-10-18T16:00:54.965 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:54.968 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:54.968 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.82 2023-10-18T16:00:55.006 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:55.009 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:55.009 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.83 2023-10-18T16:00:55.057 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:55.059 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:55.060 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.84 2023-10-18T16:00:55.106 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:55.109 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:55.109 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.85 2023-10-18T16:00:55.149 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:55.151 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:55.151 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.86 2023-10-18T16:00:55.189 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:55.191 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:55.192 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.87 2023-10-18T16:00:55.226 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:55.228 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:55.228 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.88 2023-10-18T16:00:55.266 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:55.269 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:55.269 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.89 2023-10-18T16:00:55.304 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:55.306 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:55.307 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.90 2023-10-18T16:00:55.341 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:55.344 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:55.344 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.91 2023-10-18T16:00:55.379 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:55.381 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:55.381 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.92 2023-10-18T16:00:55.419 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:55.421 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:55.422 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.93 2023-10-18T16:00:55.458 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:55.460 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:55.460 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.94 2023-10-18T16:00:55.496 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:55.498 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:55.499 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.95 2023-10-18T16:00:55.536 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:55.538 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:55.539 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.96 2023-10-18T16:00:55.590 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:55.593 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:55.593 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.97 2023-10-18T16:00:55.633 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:55.635 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:55.636 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.98 2023-10-18T16:00:55.680 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:55.682 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:55.683 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.99 2023-10-18T16:00:55.718 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:00:55.721 INFO:tasks.workunit.client.0.smithi082.stderr:++ seq -w 00 99 2023-10-18T16:00:55.722 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:55.722 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.00 --image-format 2 -s 1 2023-10-18T16:00:55.764 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:55.764 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.01 --image-format 2 -s 1 2023-10-18T16:00:55.808 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:55.808 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.02 --image-format 2 -s 1 2023-10-18T16:00:55.848 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:55.848 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.03 --image-format 2 -s 1 2023-10-18T16:00:55.888 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:55.888 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.04 --image-format 2 -s 1 2023-10-18T16:00:55.929 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:55.929 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.05 --image-format 2 -s 1 2023-10-18T16:00:55.973 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:55.973 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.06 --image-format 2 -s 1 2023-10-18T16:00:56.017 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:56.017 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.07 --image-format 2 -s 1 2023-10-18T16:00:56.056 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:56.056 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.08 --image-format 2 -s 1 2023-10-18T16:00:56.107 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:56.107 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.09 --image-format 2 -s 1 2023-10-18T16:00:56.149 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:56.149 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.10 --image-format 2 -s 1 2023-10-18T16:00:56.195 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:56.195 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.11 --image-format 2 -s 1 2023-10-18T16:00:56.236 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:56.237 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.12 --image-format 2 -s 1 2023-10-18T16:00:56.279 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:56.279 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.13 --image-format 2 -s 1 2023-10-18T16:00:56.320 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:56.321 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.14 --image-format 2 -s 1 2023-10-18T16:00:56.362 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:56.363 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.15 --image-format 2 -s 1 2023-10-18T16:00:56.402 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:56.403 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.16 --image-format 2 -s 1 2023-10-18T16:00:56.446 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:56.447 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.17 --image-format 2 -s 1 2023-10-18T16:00:56.490 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:56.490 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.18 --image-format 2 -s 1 2023-10-18T16:00:56.532 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:56.532 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.19 --image-format 2 -s 1 2023-10-18T16:00:56.570 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:56.571 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.20 --image-format 2 -s 1 2023-10-18T16:00:56.614 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:56.615 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.21 --image-format 2 -s 1 2023-10-18T16:00:56.669 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:56.669 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.22 --image-format 2 -s 1 2023-10-18T16:00:56.725 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:56.726 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.23 --image-format 2 -s 1 2023-10-18T16:00:56.766 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:56.766 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.24 --image-format 2 -s 1 2023-10-18T16:00:56.809 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:56.809 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.25 --image-format 2 -s 1 2023-10-18T16:00:56.850 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:56.850 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.26 --image-format 2 -s 1 2023-10-18T16:00:56.890 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:56.891 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.27 --image-format 2 -s 1 2023-10-18T16:00:57.132 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:57.133 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.28 --image-format 2 -s 1 2023-10-18T16:00:57.175 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:57.175 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.29 --image-format 2 -s 1 2023-10-18T16:00:57.214 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:57.214 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.30 --image-format 2 -s 1 2023-10-18T16:00:57.262 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:57.263 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.31 --image-format 2 -s 1 2023-10-18T16:00:57.307 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:57.307 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.32 --image-format 2 -s 1 2023-10-18T16:00:57.362 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:57.363 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.33 --image-format 2 -s 1 2023-10-18T16:00:57.388 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:57.389 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.34 --image-format 2 -s 1 2023-10-18T16:00:57.435 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:57.435 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.35 --image-format 2 -s 1 2023-10-18T16:00:57.475 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:57.476 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.36 --image-format 2 -s 1 2023-10-18T16:00:57.520 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:57.521 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.37 --image-format 2 -s 1 2023-10-18T16:00:57.562 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:57.562 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.38 --image-format 2 -s 1 2023-10-18T16:00:57.602 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:57.602 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.39 --image-format 2 -s 1 2023-10-18T16:00:57.641 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:57.642 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.40 --image-format 2 -s 1 2023-10-18T16:00:57.681 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:57.681 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.41 --image-format 2 -s 1 2023-10-18T16:00:57.721 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:57.722 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.42 --image-format 2 -s 1 2023-10-18T16:00:57.762 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:57.763 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.43 --image-format 2 -s 1 2023-10-18T16:00:57.803 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:57.803 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.44 --image-format 2 -s 1 2023-10-18T16:00:57.844 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:57.844 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.45 --image-format 2 -s 1 2023-10-18T16:00:57.893 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:57.894 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.46 --image-format 2 -s 1 2023-10-18T16:00:57.936 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:57.936 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.47 --image-format 2 -s 1 2023-10-18T16:00:57.976 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:57.976 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.48 --image-format 2 -s 1 2023-10-18T16:00:58.018 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:58.019 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.49 --image-format 2 -s 1 2023-10-18T16:00:58.059 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:58.059 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.50 --image-format 2 -s 1 2023-10-18T16:00:58.098 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:58.098 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.51 --image-format 2 -s 1 2023-10-18T16:00:58.139 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:58.139 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.52 --image-format 2 -s 1 2023-10-18T16:00:58.180 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:58.180 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.53 --image-format 2 -s 1 2023-10-18T16:00:58.220 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:58.221 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.54 --image-format 2 -s 1 2023-10-18T16:00:58.261 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:58.262 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.55 --image-format 2 -s 1 2023-10-18T16:00:58.301 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:58.301 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.56 --image-format 2 -s 1 2023-10-18T16:00:58.340 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:58.340 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.57 --image-format 2 -s 1 2023-10-18T16:00:58.387 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:58.387 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.58 --image-format 2 -s 1 2023-10-18T16:00:58.431 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:58.431 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.59 --image-format 2 -s 1 2023-10-18T16:00:58.474 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:58.475 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.60 --image-format 2 -s 1 2023-10-18T16:00:58.517 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:58.517 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.61 --image-format 2 -s 1 2023-10-18T16:00:58.560 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:58.560 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.62 --image-format 2 -s 1 2023-10-18T16:00:58.603 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:58.603 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.63 --image-format 2 -s 1 2023-10-18T16:00:58.642 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:58.642 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.64 --image-format 2 -s 1 2023-10-18T16:00:58.690 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:58.691 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.65 --image-format 2 -s 1 2023-10-18T16:00:58.734 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:58.734 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.66 --image-format 2 -s 1 2023-10-18T16:00:58.774 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:58.774 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.67 --image-format 2 -s 1 2023-10-18T16:00:58.813 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:58.813 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.68 --image-format 2 -s 1 2023-10-18T16:00:58.854 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:58.854 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.69 --image-format 2 -s 1 2023-10-18T16:00:58.893 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:58.893 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.70 --image-format 2 -s 1 2023-10-18T16:00:58.936 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:58.936 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.71 --image-format 2 -s 1 2023-10-18T16:00:58.978 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:58.979 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.72 --image-format 2 -s 1 2023-10-18T16:00:59.022 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:59.022 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.73 --image-format 2 -s 1 2023-10-18T16:00:59.063 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:59.063 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.74 --image-format 2 -s 1 2023-10-18T16:00:59.104 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:59.104 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.75 --image-format 2 -s 1 2023-10-18T16:00:59.143 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:59.144 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.76 --image-format 2 -s 1 2023-10-18T16:00:59.185 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:59.185 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.77 --image-format 2 -s 1 2023-10-18T16:00:59.225 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:59.225 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.78 --image-format 2 -s 1 2023-10-18T16:00:59.265 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:59.265 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.79 --image-format 2 -s 1 2023-10-18T16:00:59.306 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:59.306 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.80 --image-format 2 -s 1 2023-10-18T16:00:59.345 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:59.345 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.81 --image-format 2 -s 1 2023-10-18T16:00:59.385 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:59.385 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.82 --image-format 2 -s 1 2023-10-18T16:00:59.424 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:59.425 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.83 --image-format 2 -s 1 2023-10-18T16:00:59.465 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:59.465 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.84 --image-format 2 -s 1 2023-10-18T16:00:59.505 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:59.505 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.85 --image-format 2 -s 1 2023-10-18T16:00:59.544 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:59.544 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.86 --image-format 2 -s 1 2023-10-18T16:00:59.584 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:59.584 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.87 --image-format 2 -s 1 2023-10-18T16:00:59.624 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:59.624 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.88 --image-format 2 -s 1 2023-10-18T16:00:59.662 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:59.663 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.89 --image-format 2 -s 1 2023-10-18T16:00:59.701 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:59.702 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.90 --image-format 2 -s 1 2023-10-18T16:00:59.746 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:59.747 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.91 --image-format 2 -s 1 2023-10-18T16:00:59.790 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:59.790 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.92 --image-format 2 -s 1 2023-10-18T16:00:59.831 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:59.832 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.93 --image-format 2 -s 1 2023-10-18T16:00:59.873 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:59.874 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.94 --image-format 2 -s 1 2023-10-18T16:00:59.917 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:59.918 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.95 --image-format 2 -s 1 2023-10-18T16:00:59.959 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:00:59.959 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.96 --image-format 2 -s 1 2023-10-18T16:01:00.002 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:00.002 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.97 --image-format 2 -s 1 2023-10-18T16:01:00.044 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:00.044 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.98 --image-format 2 -s 1 2023-10-18T16:01:00.085 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:00.085 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create image.99 --image-format 2 -s 1 2023-10-18T16:01:00.130 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:01:00.130 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:01:00.130 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 100 2023-10-18T16:01:00.156 INFO:tasks.workunit.client.0.smithi082.stdout:100 2023-10-18T16:01:00.157 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls -l 2023-10-18T16:01:00.157 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep image 2023-10-18T16:01:00.158 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:01:00.158 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 100 2023-10-18T16:01:00.204 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:00.200+0000 7f1a80d06700 0 -- 172.21.15.82:0/850860378 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x55a9d42a53f0 msgr2=0x55a9d42adbf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-18T16:01:00.215 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:00.212+0000 7f1a81507700 0 -- 172.21.15.82:0/850860378 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x55a9d4293310 msgr2=0x55a9d428c760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:00.466 INFO:tasks.workunit.client.0.smithi082.stdout:100 2023-10-18T16:01:00.467 INFO:tasks.workunit.client.0.smithi082.stderr:++ seq -w 00 99 2023-10-18T16:01:00.467 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:00.468 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.00 2023-10-18T16:01:00.571 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:00.573 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:00.573 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.01 2023-10-18T16:01:00.661 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:00.664 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:00.664 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.02 2023-10-18T16:01:00.765 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:00.767 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:00.767 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.03 2023-10-18T16:01:00.876 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:00.878 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:00.878 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.04 2023-10-18T16:01:00.962 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:00.964 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:00.964 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.05 2023-10-18T16:01:01.059 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:01.062 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:01.062 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.06 2023-10-18T16:01:01.175 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:01.177 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:01.178 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.07 2023-10-18T16:01:01.267 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:01.263+0000 7f5be5f9c700 0 -- 172.21.15.82:0/2279607359 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7f5bcc008780 msgr2=0x7f5bcc00abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:01.267 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:01.269 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:01.270 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.08 2023-10-18T16:01:01.357 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:01.360 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:01.360 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.09 2023-10-18T16:01:01.470 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:01.473 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:01.473 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.10 2023-10-18T16:01:01.570 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:01.572 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:01.572 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.11 2023-10-18T16:01:01.654 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:01.657 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:01.657 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.12 2023-10-18T16:01:01.759 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:01.762 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:01.762 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.13 2023-10-18T16:01:01.857 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:01.852+0000 7f98cc409700 0 -- 172.21.15.82:0/3092242349 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x7f98b0036470 msgr2=0x7f98b00388a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:01.858 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:01.861 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:01.861 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.14 2023-10-18T16:01:01.966 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:01.969 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:01.969 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.15 2023-10-18T16:01:02.073 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:02.075 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:02.076 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.16 2023-10-18T16:01:02.164 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:02.166 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:02.167 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.17 2023-10-18T16:01:02.261 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:02.262 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:02.263 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.18 2023-10-18T16:01:02.422 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:02.425 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:02.425 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.19 2023-10-18T16:01:02.526 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:02.522+0000 7fe88450e700 0 -- 172.21.15.82:0/1051363305 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7fe8680363a0 msgr2=0x7fe8680387d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:02.528 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:02.531 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:02.531 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.20 2023-10-18T16:01:02.691 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:02.687+0000 7f0d3e730700 0 -- 172.21.15.82:0/2132234455 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x564eb3491520 msgr2=0x564eb34a36c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:02.693 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:02.695 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:02.696 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.21 2023-10-18T16:01:02.789 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:02.792 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:02.793 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.22 2023-10-18T16:01:02.905 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:02.908 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:02.908 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.23 2023-10-18T16:01:03.015 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:03.011+0000 7f52960a8700 0 -- 172.21.15.82:0/1120647584 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x56021a805520 msgr2=0x56021a81a230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:03.016 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:03.019 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:03.019 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.24 2023-10-18T16:01:03.110 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:03.113 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:03.113 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.25 2023-10-18T16:01:03.215 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:03.217 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:03.218 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.26 2023-10-18T16:01:03.319 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:03.315+0000 7fd8d4e16700 0 -- 172.21.15.82:0/1569283155 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x561c9210b310 msgr2=0x561c92151010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:03.320 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:03.322 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:03.322 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.27 2023-10-18T16:01:03.437 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:03.440 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:03.440 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.28 2023-10-18T16:01:03.530 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:03.526+0000 7fe6aa641700 0 -- 172.21.15.82:0/1628001929 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7fe68c0363a0 msgr2=0x7fe68c0387d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:03.534 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:03.537 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:03.537 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.29 2023-10-18T16:01:03.664 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:03.667 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:03.667 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.30 2023-10-18T16:01:03.768 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:03.770 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:03.771 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.31 2023-10-18T16:01:03.896 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:03.898 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:03.899 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.32 2023-10-18T16:01:03.991 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:03.993 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:03.993 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.33 2023-10-18T16:01:04.098 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:04.101 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:04.101 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.34 2023-10-18T16:01:04.190 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:04.186+0000 7f85f6a95700 0 -- 172.21.15.82:0/677177707 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x55bbfaa48a40 msgr2=0x55bbfaaecdd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:04.191 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:04.194 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:04.194 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.35 2023-10-18T16:01:04.285 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:04.281+0000 7fe105b27700 0 -- 172.21.15.82:0/1200278256 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x5607130fc520 msgr2=0x7fe0e803b4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:04.296 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:04.299 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:04.299 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.36 2023-10-18T16:01:04.402 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:04.405 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:04.405 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.37 2023-10-18T16:01:04.500 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:04.502 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:04.502 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.38 2023-10-18T16:01:04.605 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:04.608 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:04.608 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.39 2023-10-18T16:01:04.710 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:04.713 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:04.713 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.40 2023-10-18T16:01:04.804 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:04.805 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:04.805 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.41 2023-10-18T16:01:04.909 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:04.912 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:04.912 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.42 2023-10-18T16:01:05.001 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:05.003 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:05.004 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.43 2023-10-18T16:01:05.107 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:05.103+0000 7ff2def59700 0 -- 172.21.15.82:0/3916297028 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x7ff2c0036350 msgr2=0x7ff2c0038780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:05.112 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:05.115 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:05.115 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.44 2023-10-18T16:01:05.226 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:05.229 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:05.229 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.45 2023-10-18T16:01:05.330 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:05.332 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:05.332 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.46 2023-10-18T16:01:05.426 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:05.428 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:05.428 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.47 2023-10-18T16:01:05.546 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:05.549 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:05.549 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.48 2023-10-18T16:01:05.640 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:05.636+0000 7fd9109d9700 0 -- 172.21.15.82:0/3280997408 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7fd8f40363a0 msgr2=0x7fd8f40387d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-18T16:01:05.642 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:05.645 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:05.645 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.49 2023-10-18T16:01:05.739 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:05.735+0000 7f9387f00700 0 -- 172.21.15.82:0/3287555289 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7f9364008780 msgr2=0x7f936400abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:05.739 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:05.741 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:05.742 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.50 2023-10-18T16:01:05.857 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:05.860 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:05.860 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.51 2023-10-18T16:01:05.945 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:05.948 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:05.948 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.52 2023-10-18T16:01:06.074 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:06.075 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:06.075 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.53 2023-10-18T16:01:06.176 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:06.179 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:06.179 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.54 2023-10-18T16:01:06.265 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:06.268 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:06.268 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.55 2023-10-18T16:01:06.372 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:06.375 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:06.375 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.56 2023-10-18T16:01:06.478 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:06.480 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:06.481 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.57 2023-10-18T16:01:06.584 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:06.586 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:06.587 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.58 2023-10-18T16:01:06.699 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:06.694+0000 7f638ee6d700 0 -- 172.21.15.82:0/4211642392 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x55ce3a956c60 msgr2=0x55ce3a9fc290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-18T16:01:06.704 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:06.707 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:06.707 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.59 2023-10-18T16:01:06.814 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:06.815 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:06.816 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.60 2023-10-18T16:01:06.902 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:06.905 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:06.905 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.61 2023-10-18T16:01:07.018 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:07.021 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:07.021 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.62 2023-10-18T16:01:07.128 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:07.131 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:07.131 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.63 2023-10-18T16:01:07.234 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:07.237 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:07.237 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.64 2023-10-18T16:01:07.329 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:07.332 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:07.332 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.65 2023-10-18T16:01:07.455 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:07.456 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:07.457 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.66 2023-10-18T16:01:07.541 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:07.544 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:07.544 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.67 2023-10-18T16:01:07.652 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:07.654 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:07.655 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.68 2023-10-18T16:01:07.758 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:07.760 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:07.761 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.69 2023-10-18T16:01:07.852 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:07.855 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:07.855 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.70 2023-10-18T16:01:07.964 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:07.967 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:07.967 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.71 2023-10-18T16:01:08.056 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:08.052+0000 7ff26adce700 0 -- 172.21.15.82:0/3262950131 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7ff2500363a0 msgr2=0x7ff2500387d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-18T16:01:08.065 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:08.068 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:08.068 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.72 2023-10-18T16:01:08.222 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:08.224 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:08.225 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.73 2023-10-18T16:01:08.329 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:08.332 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:08.332 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.74 2023-10-18T16:01:08.444 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:08.440+0000 7fcf0f734700 0 -- 172.21.15.82:0/316597643 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7fceec008780 msgr2=0x7fceec00abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:08.444 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:08.446 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:08.446 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.75 2023-10-18T16:01:08.535 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:08.538 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:08.538 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.76 2023-10-18T16:01:08.641 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:08.644 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:08.644 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.77 2023-10-18T16:01:08.729 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:08.731 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:08.732 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.78 2023-10-18T16:01:08.846 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:08.849 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:08.849 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.79 2023-10-18T16:01:08.937 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:08.939 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:08.939 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.80 2023-10-18T16:01:09.034 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:09.037 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:09.038 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.81 2023-10-18T16:01:09.139 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:09.141 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:09.141 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.82 2023-10-18T16:01:09.282 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:09.285 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:09.285 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.83 2023-10-18T16:01:09.395 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:09.397 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:09.398 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.84 2023-10-18T16:01:09.484 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:09.487 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:09.487 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.85 2023-10-18T16:01:09.590 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:09.592 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:09.593 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.86 2023-10-18T16:01:09.700 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:09.696+0000 7fccdef18700 0 -- 172.21.15.82:0/2889377948 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x7fccc00363a0 msgr2=0x7fccc00387d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:09.903 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:09.906 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:09.907 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.87 2023-10-18T16:01:10.207 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:10.203+0000 7f0079c90700 0 -- 172.21.15.82:0/125813675 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x7f005c0363a0 msgr2=0x7f005c0387d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:10.211 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:10.214 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:10.214 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.88 2023-10-18T16:01:10.323 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:10.325 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:10.326 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.89 2023-10-18T16:01:10.672 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:10.668+0000 7feed8ae1700 0 -- 172.21.15.82:0/103870830 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x55a2b71ea0d0 msgr2=0x55a2b71c60d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:10.672 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:10.674 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:10.674 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.90 2023-10-18T16:01:10.780 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:10.776+0000 7fe2dc47d700 0 -- 172.21.15.82:0/2646944165 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x55c981f06f10 msgr2=0x55c981f06090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:10.780 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:10.782 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:10.782 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.91 2023-10-18T16:01:10.890 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:10.892 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:10.893 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.92 2023-10-18T16:01:10.983 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:10.986 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:10.986 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.93 2023-10-18T16:01:11.093 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:11.089+0000 7f982f4b0700 0 -- 172.21.15.82:0/4114805681 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x557473aa4350 msgr2=0x557473ab5f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-18T16:01:11.097 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:11.100 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:11.100 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.94 2023-10-18T16:01:11.204 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:11.207 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:11.207 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.95 2023-10-18T16:01:11.321 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:11.317+0000 7fad239fb700 0 -- 172.21.15.82:0/713049266 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7fad00008780 msgr2=0x7fad0000abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:11.321 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:11.324 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:11.324 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.96 2023-10-18T16:01:11.428 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:11.424+0000 7fbfe13fe700 0 -- 172.21.15.82:0/712077259 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x556a1d583a40 msgr2=0x556a1d6290d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:11.431 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:11.434 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:11.434 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.97 2023-10-18T16:01:11.550 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:11.552 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:11.552 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.98 2023-10-18T16:01:11.646 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:11.648 INFO:tasks.workunit.client.0.smithi082.stderr:+ for i in $(seq -w 00 99) 2023-10-18T16:01:11.649 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm image.99 2023-10-18T16:01:11.763 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:11.766 INFO:tasks.workunit.client.0.smithi082.stderr:+ test_remove 2023-10-18T16:01:11.766 INFO:tasks.workunit.client.0.smithi082.stdout:testing remove... 2023-10-18T16:01:11.767 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'testing remove...' 2023-10-18T16:01:11.767 INFO:tasks.workunit.client.0.smithi082.stderr:+ remove_images 2023-10-18T16:01:11.767 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:11.841 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:11.911 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:11.982 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:12.052 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:12.325 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:12.400 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:12.473 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:12.544 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:12.622 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:12.693 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:12.764 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:12.836 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:12.909 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:12.984 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:13.060 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:13.136 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:13.213 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:13.307 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd remove NOT_EXIST 2023-10-18T16:01:13.365 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 0% complete...failed. 2023-10-18T16:01:13.365 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: delete error: (2) No such file or directory 2023-10-18T16:01:13.367 INFO:tasks.workunit.client.0.smithi082.stderr:+ true 2023-10-18T16:01:13.368 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 1 -s 1 test1 2023-10-18T16:01:13.390 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: image format 1 is deprecated 2023-10-18T16:01:13.395 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:13.392+0000 7fadcf594380 -1 librbd: Forced V1 image creation. 2023-10-18T16:01:13.403 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test1 2023-10-18T16:01:13.441 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:13.443 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:01:13.444 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:01:13.444 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^0$' 2023-10-18T16:01:13.469 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:01:13.469 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 -s 1 test2 2023-10-18T16:01:13.512 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test2 2023-10-18T16:01:13.626 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:13.629 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:01:13.629 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:01:13.629 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^0$' 2023-10-18T16:01:13.654 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:01:13.654 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 1 -s 1 test1 2023-10-18T16:01:13.676 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: image format 1 is deprecated 2023-10-18T16:01:13.682 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:13.678+0000 7f48ce40e380 -1 librbd: Forced V1 image creation. 2023-10-18T16:01:13.688 INFO:tasks.workunit.client.0.smithi082.stderr:+ rados rm -p rbd test1.rbd 2023-10-18T16:01:13.716 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test1 2023-10-18T16:01:13.745 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:13.748 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:01:13.748 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:01:13.748 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^0$' 2023-10-18T16:01:13.773 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:01:13.773 INFO:tasks.workunit.client.0.smithi082.stderr:+ '[' 1 -eq 0 ']' 2023-10-18T16:01:13.774 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 -s 1 test2 2023-10-18T16:01:13.818 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create test2@snap 2023-10-18T16:01:13.959 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:01:13.968 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap protect test2@snap 2023-10-18T16:01:14.008 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone test2@snap clone --rbd-default-clone-format 1 2023-10-18T16:01:14.076 INFO:tasks.workunit.client.0.smithi082.stderr:+ rados -p rbd rm rbd_children 2023-10-18T16:01:14.106 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm clone 2023-10-18T16:01:14.221 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:14.225 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:01:14.225 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep clone 2023-10-18T16:01:14.225 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:01:14.225 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^0$' 2023-10-18T16:01:14.250 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:01:14.250 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap unprotect test2@snap 2023-10-18T16:01:14.307 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap rm test2@snap 2023-10-18T16:01:14.965 INFO:tasks.workunit.client.0.smithi082.stderr: Removing snap: 100% complete...done. 2023-10-18T16:01:14.975 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test2 2023-10-18T16:01:15.081 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:15.083 INFO:tasks.workunit.client.0.smithi082.stderr:+ test_migration 2023-10-18T16:01:15.084 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'testing migration...' 2023-10-18T16:01:15.084 INFO:tasks.workunit.client.0.smithi082.stdout:testing migration... 2023-10-18T16:01:15.085 INFO:tasks.workunit.client.0.smithi082.stderr:+ remove_images 2023-10-18T16:01:15.085 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:15.157 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:15.231 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:15.302 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:15.377 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:15.452 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:15.533 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:15.606 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:15.679 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:15.752 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:15.824 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:15.894 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:15.966 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:16.037 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:16.115 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:16.191 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:16.267 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:16.338 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:16.410 INFO:tasks.workunit.client.0.smithi082.stderr:+ ceph osd pool create rbd2 8 2023-10-18T16:01:17.072 INFO:tasks.workunit.client.0.smithi082.stderr:pool 'rbd2' already exists 2023-10-18T16:01:17.081 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd pool init rbd2 2023-10-18T16:01:19.910 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 1 -s 128M test1 2023-10-18T16:01:19.934 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: image format 1 is deprecated 2023-10-18T16:01:19.941 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:19.937+0000 7fcf9d642380 -1 librbd: Forced V1 image creation. 2023-10-18T16:01:19.950 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info test1 2023-10-18T16:01:19.950 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'format: 1' 2023-10-18T16:01:19.985 INFO:tasks.workunit.client.0.smithi082.stdout: format: 1 2023-10-18T16:01:19.986 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration prepare test1 --image-format 2 2023-10-18T16:01:20.054 INFO:tasks.workunit.client.0.smithi082.stderr:++ get_migration_state test1 2023-10-18T16:01:20.055 INFO:tasks.workunit.client.0.smithi082.stderr:++ local image=test1 2023-10-18T16:01:20.055 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd --format xml status test1 2023-10-18T16:01:20.055 INFO:tasks.workunit.client.0.smithi082.stderr:++ xmlstarlet sel -t -v //status/migration/state 2023-10-18T16:01:20.127 INFO:tasks.workunit.client.0.smithi082.stderr:+ test prepared = prepared 2023-10-18T16:01:20.127 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info test1 2023-10-18T16:01:20.127 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'format: 2' 2023-10-18T16:01:20.162 INFO:tasks.workunit.client.0.smithi082.stdout: format: 2 2023-10-18T16:01:20.163 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test1 2023-10-18T16:01:20.196 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:20.192+0000 7f99b00bf380 -1 librbd::image::PreRemoveRequest: 0x55a2f4a18510 validate_image_removal: image in migration state - not removing 2023-10-18T16:01:20.198 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 0% complete...failed. 2023-10-18T16:01:20.198 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: error: image still has watchers 2023-10-18T16:01:20.198 INFO:tasks.workunit.client.0.smithi082.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2023-10-18T16:01:20.200 INFO:tasks.workunit.client.0.smithi082.stderr:+ true 2023-10-18T16:01:20.200 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration execute test1 2023-10-18T16:01:20.306 INFO:tasks.workunit.client.0.smithi082.stderr: Image migration: 3% complete... Image migration: 6% complete... Image migration: 9% complete... Image migration: 12% complete... Image migration: 15% complete... Image migration: 18% complete... Image migration: 21% complete... Image migration: 25% complete... Image migration: 28% complete... Image migration: 31% complete... Image migration: 34% complete... Image migration: 37% complete... Image migration: 40% complete... Image migration: 43% complete... Image migration: 46% complete... Image migration: 50% complete... Image migration: 53% complete... Image migration: 56% complete... Image migration: 59% complete... Image migration: 62% complete... Image migration: 65% complete... Image migration: 68% complete... Image migration: 71% complete... Image migration: 75% complete... Image migration: 78% complete... Image migration: 81% complete... Image migration: 84% complete... Image migration: 87% complete... Image migration: 90% complete... Image migration: 93% complete... Image migration: 96% complete...2023-10-18T16:01:20.302+0000 7f877950b700 0 -- 172.21.15.82:0/3912114524 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x556c11e76320 msgr2=0x556c11f62a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:20.331 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:20.328+0000 7f877950b700 0 -- 172.21.15.82:0/3912114524 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7f875c036a10 msgr2=0x7f875c038e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:20.345 INFO:tasks.workunit.client.0.smithi082.stderr: Image migration: 100% complete...done. 2023-10-18T16:01:20.349 INFO:tasks.workunit.client.0.smithi082.stderr:++ get_migration_state test1 2023-10-18T16:01:20.349 INFO:tasks.workunit.client.0.smithi082.stderr:++ local image=test1 2023-10-18T16:01:20.349 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd --format xml status test1 2023-10-18T16:01:20.349 INFO:tasks.workunit.client.0.smithi082.stderr:++ xmlstarlet sel -t -v //status/migration/state 2023-10-18T16:01:20.412 INFO:tasks.workunit.client.0.smithi082.stderr:+ test executed = executed 2023-10-18T16:01:20.413 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration commit test1 2023-10-18T16:01:20.504 INFO:tasks.workunit.client.0.smithi082.stderr: Commit image migration: 3% complete... Commit image migration: 6% complete... Commit image migration: 9% complete... Commit image migration: 12% complete... Commit image migration: 15% complete... Commit image migration: 18% complete... Commit image migration: 21% complete... Commit image migration: 25% complete... Commit image migration: 28% complete... Commit image migration: 31% complete... Commit image migration: 34% complete... Commit image migration: 37% complete... Commit image migration: 40% complete... Commit image migration: 43% complete... Commit image migration: 46% complete... Commit image migration: 50% complete... Commit image migration: 53% complete... Commit image migration: 56% complete...2023-10-18T16:01:20.500+0000 7fbbaf0e3700 0 -- 172.21.15.82:0/1708149332 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x55cff73857b0 msgr2=0x55cff7471aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-18T16:01:20.515 INFO:tasks.workunit.client.0.smithi082.stderr: Commit image migration: 59% complete... Commit image migration: 62% complete... Commit image migration: 65% complete... Commit image migration: 68% complete... Commit image migration: 71% complete... Commit image migration: 75% complete... Commit image migration: 78% complete... Commit image migration: 81% complete... Commit image migration: 84% complete... Commit image migration: 87% complete... Commit image migration: 90% complete... Commit image migration: 93% complete... Commit image migration: 96% complete...2023-10-18T16:01:20.511+0000 7fbbaf0e3700 0 -- 172.21.15.82:0/1708149332 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7fbb94036ae0 msgr2=0x7fbb94038f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:20.527 INFO:tasks.workunit.client.0.smithi082.stderr: Commit image migration: 100% complete...done. 2023-10-18T16:01:20.530 INFO:tasks.workunit.client.0.smithi082.stderr:+ get_migration_state test1 2023-10-18T16:01:20.530 INFO:tasks.workunit.client.0.smithi082.stderr:+ local image=test1 2023-10-18T16:01:20.530 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd --format xml status test1 2023-10-18T16:01:20.530 INFO:tasks.workunit.client.0.smithi082.stderr:+ xmlstarlet sel -t -v //status/migration/state 2023-10-18T16:01:20.564 INFO:tasks.workunit.client.0.smithi082.stderr:+ true 2023-10-18T16:01:20.564 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info test1 2023-10-18T16:01:20.564 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'features: .*layering' 2023-10-18T16:01:20.597 INFO:tasks.workunit.client.0.smithi082.stderr:+ true 2023-10-18T16:01:20.597 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration prepare test1 --image-feature layering,exclusive-lock,object-map,fast-diff,deep-flatten 2023-10-18T16:01:20.692 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info test1 2023-10-18T16:01:20.692 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'features: .*layering' 2023-10-18T16:01:20.730 INFO:tasks.workunit.client.0.smithi082.stdout: features: layering, exclusive-lock, object-map, fast-diff, deep-flatten, migrating 2023-10-18T16:01:20.731 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration execute test1 2023-10-18T16:01:20.813 INFO:tasks.workunit.client.0.smithi082.stderr: Image migration: 3% complete... Image migration: 6% complete... Image migration: 9% complete... Image migration: 12% complete... Image migration: 15% complete... Image migration: 18% complete... Image migration: 21% complete... Image migration: 25% complete... Image migration: 28% complete... Image migration: 31% complete... Image migration: 34% complete... Image migration: 37% complete... Image migration: 40% complete... Image migration: 43% complete... Image migration: 46% complete... Image migration: 50% complete... Image migration: 53% complete... Image migration: 56% complete... Image migration: 59% complete... Image migration: 62% complete... Image migration: 65% complete... Image migration: 68% complete... Image migration: 71% complete... Image migration: 75% complete... Image migration: 78% complete... Image migration: 81% complete... Image migration: 84% complete... Image migration: 87% complete... Image migration: 90% complete... Image migration: 93% complete... Image migration: 96% complete...2023-10-18T16:01:20.809+0000 7f7cffac6700 0 -- 172.21.15.82:0/178242826 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x7f7cdc008780 msgr2=0x7f7cdc00abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:20.838 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:20.834+0000 7f7d002c7700 0 -- 172.21.15.82:0/178242826 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x556270e88540 msgr2=0x556270f74a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:20.838 INFO:tasks.workunit.client.0.smithi082.stderr: Image migration: 100% complete...done. 2023-10-18T16:01:20.840 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration commit test1 2023-10-18T16:01:20.927 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:20.923+0000 7f3d15933700 0 -- 172.21.15.82:0/621901981 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x7f3cf4008780 msgr2=0x7f3cf400abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:20.938 INFO:tasks.workunit.client.0.smithi082.stderr: Commit image migration: 3% complete... Commit image migration: 6% complete... Commit image migration: 9% complete... Commit image migration: 12% complete... Commit image migration: 15% complete... Commit image migration: 18% complete... Commit image migration: 21% complete... Commit image migration: 25% complete... Commit image migration: 28% complete... Commit image migration: 31% complete... Commit image migration: 34% complete... Commit image migration: 37% complete... Commit image migration: 40% complete... Commit image migration: 43% complete... Commit image migration: 46% complete... Commit image migration: 50% complete... Commit image migration: 53% complete... Commit image migration: 56% complete... Commit image migration: 59% complete...2023-10-18T16:01:20.934+0000 7f3d15933700 0 -- 172.21.15.82:0/621901981 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x7f3cf4008780 msgr2=0x7f3cf8039410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:21.157 INFO:tasks.workunit.client.0.smithi082.stderr: Commit image migration: 62% complete... Commit image migration: 65% complete... Commit image migration: 68% complete... Commit image migration: 71% complete... Commit image migration: 75% complete... Commit image migration: 78% complete... Commit image migration: 81% complete... Commit image migration: 84% complete... Commit image migration: 87% complete... Commit image migration: 90% complete... Commit image migration: 93% complete... Commit image migration: 96% complete... Commit image migration: 100% complete...done. 2023-10-18T16:01:21.160 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration prepare test1 rbd2/test1 2023-10-18T16:01:21.249 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:21.244+0000 7fcb018cf700 0 -- 172.21.15.82:0/3193776159 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x7fcae8008780 msgr2=0x7fcae800abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:21.251 INFO:tasks.workunit.client.0.smithi082.stderr:++ get_migration_state rbd2/test1 2023-10-18T16:01:21.251 INFO:tasks.workunit.client.0.smithi082.stderr:++ local image=rbd2/test1 2023-10-18T16:01:21.251 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd --format xml status rbd2/test1 2023-10-18T16:01:21.252 INFO:tasks.workunit.client.0.smithi082.stderr:++ xmlstarlet sel -t -v //status/migration/state 2023-10-18T16:01:21.323 INFO:tasks.workunit.client.0.smithi082.stderr:+ test prepared = prepared 2023-10-18T16:01:21.324 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:01:21.324 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:01:21.324 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^0$' 2023-10-18T16:01:21.349 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:01:21.350 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd -p rbd2 ls 2023-10-18T16:01:21.350 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test1 2023-10-18T16:01:21.376 INFO:tasks.workunit.client.0.smithi082.stdout:test1 2023-10-18T16:01:21.377 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration execute test1 2023-10-18T16:01:21.476 INFO:tasks.workunit.client.0.smithi082.stderr: Image migration: 3% complete... Image migration: 6% complete... Image migration: 9% complete... Image migration: 12% complete... Image migration: 15% complete... Image migration: 18% complete... Image migration: 21% complete... Image migration: 25% complete... Image migration: 28% complete... Image migration: 31% complete... Image migration: 34% complete... Image migration: 37% complete... Image migration: 40% complete... Image migration: 43% complete... Image migration: 46% complete... Image migration: 50% complete... Image migration: 53% complete... Image migration: 56% complete... Image migration: 59% complete... Image migration: 62% complete... Image migration: 65% complete... Image migration: 68% complete... Image migration: 71% complete... Image migration: 75% complete... Image migration: 78% complete... Image migration: 81% complete... Image migration: 84% complete... Image migration: 87% complete... Image migration: 90% complete... Image migration: 93% complete... Image migration: 96% complete...2023-10-18T16:01:21.471+0000 7f35d36c0700 0 -- 172.21.15.82:0/1326968165 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x556e738c90f0 msgr2=0x556e739d6370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:21.483 INFO:tasks.workunit.client.0.smithi082.stderr: Image migration: 100% complete...done. 2023-10-18T16:01:21.486 INFO:tasks.workunit.client.0.smithi082.stderr:++ get_migration_state rbd2/test1 2023-10-18T16:01:21.487 INFO:tasks.workunit.client.0.smithi082.stderr:++ local image=rbd2/test1 2023-10-18T16:01:21.487 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd --format xml status rbd2/test1 2023-10-18T16:01:21.487 INFO:tasks.workunit.client.0.smithi082.stderr:++ xmlstarlet sel -t -v //status/migration/state 2023-10-18T16:01:21.547 INFO:tasks.workunit.client.0.smithi082.stderr:+ test executed = executed 2023-10-18T16:01:21.548 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm rbd2/test1 2023-10-18T16:01:21.582 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:21.578+0000 7f9aed651700 -1 librbd::image::PreRemoveRequest: 0x5612737f3e50 validate_image_removal: image in migration state - not removing 2023-10-18T16:01:21.587 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 0% complete...failed. 2023-10-18T16:01:21.587 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: error: image still has watchers 2023-10-18T16:01:21.587 INFO:tasks.workunit.client.0.smithi082.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2023-10-18T16:01:21.590 INFO:tasks.workunit.client.0.smithi082.stderr:+ true 2023-10-18T16:01:21.590 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration commit test1 2023-10-18T16:01:21.673 INFO:tasks.workunit.client.0.smithi082.stderr: Commit image migration: 3% complete... Commit image migration: 6% complete... Commit image migration: 9% complete... Commit image migration: 12% complete... Commit image migration: 15% complete... Commit image migration: 18% complete... Commit image migration: 21% complete... Commit image migration: 25% complete... Commit image migration: 28% complete... Commit image migration: 31% complete... Commit image migration: 34% complete... Commit image migration: 37% complete... Commit image migration: 40% complete... Commit image migration: 43% complete... Commit image migration: 46% complete... Commit image migration: 50% complete... Commit image migration: 53% complete... Commit image migration: 56% complete... Commit image migration: 59% complete... Commit image migration: 62% complete... Commit image migration: 65% complete... Commit image migration: 68% complete... Commit image migration: 71% complete... Commit image migration: 75% complete... Commit image migration: 78% complete... Commit image migration: 81% complete... Commit image migration: 84% complete... Commit image migration: 87% complete... Commit image migration: 90% complete... Commit image migration: 93% complete... Commit image migration: 96% complete...2023-10-18T16:01:21.669+0000 7f30b5aa0700 0 -- 172.21.15.82:0/1836248892 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x7f3098036ad0 msgr2=0x7f3098038f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:21.685 INFO:tasks.workunit.client.0.smithi082.stderr: Commit image migration: 100% complete...done. 2023-10-18T16:01:21.688 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd namespace create rbd2/ns1 2023-10-18T16:01:21.717 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd namespace create rbd2/ns2 2023-10-18T16:01:21.746 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration prepare rbd2/test1 rbd2/ns1/test1 2023-10-18T16:01:21.830 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:21.826+0000 7fe661cd1700 0 -- 172.21.15.82:0/1956184194 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x561a0974f0b0 msgr2=0x561a097f0af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:21.835 INFO:tasks.workunit.client.0.smithi082.stderr:++ get_migration_state rbd2/ns1/test1 2023-10-18T16:01:21.835 INFO:tasks.workunit.client.0.smithi082.stderr:++ local image=rbd2/ns1/test1 2023-10-18T16:01:21.835 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd --format xml status rbd2/ns1/test1 2023-10-18T16:01:21.835 INFO:tasks.workunit.client.0.smithi082.stderr:++ xmlstarlet sel -t -v //status/migration/state 2023-10-18T16:01:21.905 INFO:tasks.workunit.client.0.smithi082.stderr:+ test prepared = prepared 2023-10-18T16:01:21.905 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration execute rbd2/test1 2023-10-18T16:01:21.973 INFO:tasks.workunit.client.0.smithi082.stderr: Image migration: 3% complete... Image migration: 6% complete... Image migration: 9% complete... Image migration: 12% complete... Image migration: 15% complete... Image migration: 18% complete... Image migration: 21% complete... Image migration: 25% complete... Image migration: 28% complete... Image migration: 31% complete... Image migration: 34% complete... Image migration: 37% complete... Image migration: 40% complete... Image migration: 43% complete... Image migration: 46% complete... Image migration: 50% complete... Image migration: 53% complete... Image migration: 56% complete... Image migration: 59% complete... Image migration: 62% complete... Image migration: 65% complete... Image migration: 68% complete... Image migration: 71% complete... Image migration: 75% complete... Image migration: 78% complete... Image migration: 81% complete... Image migration: 84% complete... Image migration: 87% complete... Image migration: 90% complete... Image migration: 93% complete... Image migration: 96% complete...2023-10-18T16:01:21.969+0000 7f2fdb14b700 0 -- 172.21.15.82:0/1793831351 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x55868f0eb320 msgr2=0x55868f1d7580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:21.974 INFO:tasks.workunit.client.0.smithi082.stderr: Image migration: 100% complete...done. 2023-10-18T16:01:21.977 INFO:tasks.workunit.client.0.smithi082.stderr:++ get_migration_state rbd2/ns1/test1 2023-10-18T16:01:21.977 INFO:tasks.workunit.client.0.smithi082.stderr:++ local image=rbd2/ns1/test1 2023-10-18T16:01:21.978 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd --format xml status rbd2/ns1/test1 2023-10-18T16:01:21.978 INFO:tasks.workunit.client.0.smithi082.stderr:++ xmlstarlet sel -t -v //status/migration/state 2023-10-18T16:01:22.035 INFO:tasks.workunit.client.0.smithi082.stderr:+ test executed = executed 2023-10-18T16:01:22.035 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration commit rbd2/test1 2023-10-18T16:01:22.116 INFO:tasks.workunit.client.0.smithi082.stderr: Commit image migration: 3% complete... Commit image migration: 6% complete... Commit image migration: 9% complete... Commit image migration: 12% complete... Commit image migration: 15% complete... Commit image migration: 18% complete... Commit image migration: 21% complete... Commit image migration: 25% complete... Commit image migration: 28% complete... Commit image migration: 31% complete... Commit image migration: 34% complete... Commit image migration: 37% complete... Commit image migration: 40% complete... Commit image migration: 43% complete... Commit image migration: 46% complete... Commit image migration: 50% complete... Commit image migration: 53% complete... Commit image migration: 56% complete... Commit image migration: 59% complete... Commit image migration: 62% complete... Commit image migration: 65% complete... Commit image migration: 68% complete... Commit image migration: 71% complete... Commit image migration: 75% complete... Commit image migration: 78% complete... Commit image migration: 81% complete... Commit image migration: 84% complete... Commit image migration: 87% complete... Commit image migration: 90% complete... Commit image migration: 93% complete... Commit image migration: 96% complete...2023-10-18T16:01:22.112+0000 7f9147c2c700 0 -- 172.21.15.82:0/2388973330 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x55e46c8bb1f0 msgr2=0x55e46c8bd5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:22.124 INFO:tasks.workunit.client.0.smithi082.stderr: Commit image migration: 100% complete...done. 2023-10-18T16:01:22.127 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration prepare rbd2/ns1/test1 rbd2/ns2/test1 2023-10-18T16:01:22.208 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:22.204+0000 7fb000ea9700 0 -- 172.21.15.82:0/831557121 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x556a0c1ed6d0 msgr2=0x556a0c1efac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:22.213 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration execute rbd2/ns2/test1 2023-10-18T16:01:22.284 INFO:tasks.workunit.client.0.smithi082.stderr: Image migration: 3% complete... Image migration: 6% complete... Image migration: 9% complete... Image migration: 12% complete... Image migration: 15% complete... Image migration: 18% complete... Image migration: 21% complete... Image migration: 25% complete... Image migration: 28% complete... Image migration: 31% complete... Image migration: 34% complete... Image migration: 37% complete... Image migration: 40% complete... Image migration: 43% complete... Image migration: 46% complete... Image migration: 50% complete... Image migration: 53% complete... Image migration: 56% complete... Image migration: 59% complete... Image migration: 62% complete... Image migration: 65% complete... Image migration: 68% complete... Image migration: 71% complete... Image migration: 75% complete... Image migration: 78% complete... Image migration: 81% complete... Image migration: 84% complete... Image migration: 87% complete... Image migration: 90% complete... Image migration: 93% complete... Image migration: 96% complete...2023-10-18T16:01:22.280+0000 7f5f83e86700 0 -- 172.21.15.82:0/2575617570 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x55931e9249e0 msgr2=0x55931e926dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:22.293 INFO:tasks.workunit.client.0.smithi082.stderr: Image migration: 100% complete...done. 2023-10-18T16:01:22.295 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration commit rbd2/ns2/test1 2023-10-18T16:01:22.376 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:22.372+0000 7fc66ba40700 0 -- 172.21.15.82:0/2583882574 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x55c758ad97b0 msgr2=0x55c758b802b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:22.388 INFO:tasks.workunit.client.0.smithi082.stderr: Commit image migration: 3% complete... Commit image migration: 6% complete... Commit image migration: 9% complete... Commit image migration: 12% complete... Commit image migration: 15% complete... Commit image migration: 18% complete... Commit image migration: 21% complete... Commit image migration: 25% complete... Commit image migration: 28% complete... Commit image migration: 31% complete... Commit image migration: 34% complete... Commit image migration: 37% complete... Commit image migration: 40% complete... Commit image migration: 43% complete... Commit image migration: 46% complete... Commit image migration: 50% complete... Commit image migration: 53% complete... Commit image migration: 56% complete... Commit image migration: 59% complete... Commit image migration: 62% complete... Commit image migration: 65% complete... Commit image migration: 68% complete... Commit image migration: 71% complete... Commit image migration: 75% complete... Commit image migration: 78% complete... Commit image migration: 81% complete... Commit image migration: 84% complete... Commit image migration: 87% complete... Commit image migration: 90% complete... Commit image migration: 93% complete... Commit image migration: 96% complete...2023-10-18T16:01:22.385+0000 7fc66b23f700 0 -- 172.21.15.82:0/2583882574 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x55c758bc7e70 msgr2=0x55c758bca260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:22.406 INFO:tasks.workunit.client.0.smithi082.stderr: Commit image migration: 100% complete...done. 2023-10-18T16:01:22.408 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create -s 128M test1 2023-10-18T16:01:22.435 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:22.431+0000 7f2a9c96f380 -1 librbd: Forced V1 image creation. 2023-10-18T16:01:22.441 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration prepare test1 --data-pool rbd2 2023-10-18T16:01:22.509 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info test1 2023-10-18T16:01:22.510 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'data_pool: rbd2' 2023-10-18T16:01:22.545 INFO:tasks.workunit.client.0.smithi082.stdout: data_pool: rbd2 2023-10-18T16:01:22.546 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration execute test1 2023-10-18T16:01:22.626 INFO:tasks.workunit.client.0.smithi082.stderr: Image migration: 3% complete... Image migration: 6% complete... Image migration: 9% complete... Image migration: 12% complete... Image migration: 15% complete... Image migration: 18% complete... Image migration: 21% complete... Image migration: 25% complete... Image migration: 28% complete... Image migration: 31% complete... Image migration: 34% complete... Image migration: 37% complete... Image migration: 40% complete... Image migration: 43% complete... Image migration: 46% complete... Image migration: 50% complete... Image migration: 53% complete... Image migration: 56% complete... Image migration: 59% complete... Image migration: 62% complete... Image migration: 65% complete... Image migration: 68% complete... Image migration: 71% complete... Image migration: 75% complete... Image migration: 78% complete... Image migration: 81% complete... Image migration: 84% complete... Image migration: 87% complete... Image migration: 90% complete... Image migration: 93% complete... Image migration: 96% complete...2023-10-18T16:01:22.622+0000 7f20ec743700 0 -- 172.21.15.82:0/4091416452 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x7f20c8008780 msgr2=0x7f20c800abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:22.648 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:22.644+0000 7f20ec743700 0 -- 172.21.15.82:0/4091416452 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x7f20d0036a10 msgr2=0x7f20d0038e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:22.656 INFO:tasks.workunit.client.0.smithi082.stderr: Image migration: 100% complete...done. 2023-10-18T16:01:22.659 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration commit test1 2023-10-18T16:01:22.750 INFO:tasks.workunit.client.0.smithi082.stderr: Commit image migration: 3% complete... Commit image migration: 6% complete... Commit image migration: 9% complete... Commit image migration: 12% complete... Commit image migration: 15% complete... Commit image migration: 18% complete... Commit image migration: 21% complete... Commit image migration: 25% complete... Commit image migration: 28% complete... Commit image migration: 31% complete... Commit image migration: 34% complete... Commit image migration: 37% complete... Commit image migration: 40% complete... Commit image migration: 43% complete... Commit image migration: 46% complete... Commit image migration: 50% complete... Commit image migration: 53% complete... Commit image migration: 56% complete...2023-10-18T16:01:22.746+0000 7f5bd0de0700 0 -- 172.21.15.82:0/1655430116 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x565549ca75e0 msgr2=0x565549d939c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-18T16:01:22.764 INFO:tasks.workunit.client.0.smithi082.stderr: Commit image migration: 59% complete... Commit image migration: 62% complete... Commit image migration: 65% complete... Commit image migration: 68% complete... Commit image migration: 71% complete... Commit image migration: 75% complete... Commit image migration: 78% complete... Commit image migration: 81% complete... Commit image migration: 84% complete... Commit image migration: 87% complete... Commit image migration: 90% complete... Commit image migration: 93% complete... Commit image migration: 96% complete...2023-10-18T16:01:22.760+0000 7f5bd0de0700 0 -- 172.21.15.82:0/1655430116 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7f5bb4036a10 msgr2=0x7f5bb4038e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:22.777 INFO:tasks.workunit.client.0.smithi082.stderr: Commit image migration: 100% complete...done. 2023-10-18T16:01:22.780 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration prepare test1 2023-10-18T16:01:22.861 INFO:tasks.workunit.client.0.smithi082.stderr:+ expect_fail rbd trash mv test1 2023-10-18T16:01:22.862 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv test1 2023-10-18T16:01:22.895 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:22.891+0000 7f99f8506380 -1 librbd::api::Trash: move: cannot move migrating image to trash 2023-10-18T16:01:23.489 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: deferred delete error: (16) Device or resource busy 2023-10-18T16:01:23.489 INFO:tasks.workunit.client.0.smithi082.stderr:+ return 0 2023-10-18T16:01:23.489 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd trash ls -a 2023-10-18T16:01:23.489 INFO:tasks.workunit.client.0.smithi082.stderr:++ cut -d ' ' -f 1 2023-10-18T16:01:23.490 INFO:tasks.workunit.client.0.smithi082.stderr:+ ID=19e38982f2e5 2023-10-18T16:01:23.490 INFO:tasks.workunit.client.0.smithi082.stderr:+ expect_fail rbd trash rm 19e38982f2e5 2023-10-18T16:01:23.490 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash rm 19e38982f2e5 2023-10-18T16:01:23.490 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:23.144+0000 7f766e61e700 0 --2- 172.21.15.82:0/3782097335 >> v2:172.21.15.82:3300/0 conn(0x561877353960 0x5618773ee310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2023-10-18T16:01:23.490 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:23.156+0000 7f766d61c700 -1 librbd::image::RefreshRequest: image being migrated 2023-10-18T16:01:23.491 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:23.156+0000 7f766ce1b700 -1 librbd::image::OpenRequest: failed to refresh image: (30) Read-only file system 2023-10-18T16:01:23.491 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:23.156+0000 7f766d61c700 -1 librbd::ImageState: 0x7f7648048a00 failed to open image: (30) Read-only file system 2023-10-18T16:01:23.491 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:23.156+0000 7f764f7fe700 -1 librbd::image::RemoveRequest: 0x7f7648000b80 handle_open_image: error opening image: (30) Read-only file system 2023-10-18T16:01:23.491 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: remove error: (30) Read-only file system 2023-10-18T16:01:23.491 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 0% complete...failed. 2023-10-18T16:01:23.492 INFO:tasks.workunit.client.0.smithi082.stderr:+ return 0 2023-10-18T16:01:23.492 INFO:tasks.workunit.client.0.smithi082.stderr:+ expect_fail rbd trash restore 19e38982f2e5 2023-10-18T16:01:23.492 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash restore 19e38982f2e5 2023-10-18T16:01:23.492 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: restore error: 2023-10-18T16:01:23.185+0000 7fd115fcc380 -1 librbd::api::Trash: restore: Current trash source 'migration' does not match expected: user,mirroring,unknown (4) 2023-10-18T16:01:23.492 INFO:tasks.workunit.client.0.smithi082.stderr:(22) Invalid argument 2023-10-18T16:01:23.492 INFO:tasks.workunit.client.0.smithi082.stderr:+ return 0 2023-10-18T16:01:23.493 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration abort test1 2023-10-18T16:01:23.493 INFO:tasks.workunit.client.0.smithi082.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete...2023-10-18T16:01:23.280+0000 7f699d6e3700 0 -- 172.21.15.82:0/3767265974 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x7f6980036ae0 msgr2=0x7f6980038f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:23.493 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:23.311+0000 7f699dee4700 0 -- 172.21.15.82:0/3767265974 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x564f697ca540 msgr2=0x564f698b6a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:23.493 INFO:tasks.workunit.client.0.smithi082.stderr: Abort image migration: 100% complete...done. 2023-10-18T16:01:23.493 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd remove test1 2023-10-18T16:01:23.494 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2023-10-18T16:01:23.494 INFO:tasks.workunit.client.0.smithi082.stderr:+ dd if=/dev/urandom bs=1M count=1 2023-10-18T16:01:23.494 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd --image-format 2 import - test1 2023-10-18T16:01:23.494 INFO:tasks.workunit.client.0.smithi082.stderr:1+0 records in 2023-10-18T16:01:23.494 INFO:tasks.workunit.client.0.smithi082.stderr:1+0 records out 2023-10-18T16:01:23.495 INFO:tasks.workunit.client.0.smithi082.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0514624 s, 20.4 MB/s 2023-10-18T16:01:23.506 INFO:tasks.workunit.client.0.smithi082.stderr: Importing image: 100% complete...done. 2023-10-18T16:01:23.509 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd export test1 - 2023-10-18T16:01:23.509 INFO:tasks.workunit.client.0.smithi082.stderr:++ md5sum 2023-10-18T16:01:23.542 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 100% complete...done. 2023-10-18T16:01:23.545 INFO:tasks.workunit.client.0.smithi082.stderr:+ md5sum='1addd36bf90e03ff605c772595892efd -' 2023-10-18T16:01:23.546 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create test1@snap1 2023-10-18T16:01:23.919 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:01:23.929 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap protect test1@snap1 2023-10-18T16:01:23.969 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create test1@snap2 2023-10-18T16:01:24.933 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:01:24.946 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone test1@snap1 clone_v1 --rbd_default_clone_format=1 2023-10-18T16:01:25.017 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone test1@snap2 clone_v2 --rbd_default_clone_format=2 2023-10-18T16:01:25.080 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info clone_v1 2023-10-18T16:01:25.080 INFO:tasks.workunit.client.0.smithi082.stderr:+ fgrep 'parent: rbd/test1@snap1' 2023-10-18T16:01:25.121 INFO:tasks.workunit.client.0.smithi082.stdout: parent: rbd/test1@snap1 2023-10-18T16:01:25.122 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info clone_v2 2023-10-18T16:01:25.122 INFO:tasks.workunit.client.0.smithi082.stderr:+ fgrep 'parent: rbd/test1@snap2' 2023-10-18T16:01:25.161 INFO:tasks.workunit.client.0.smithi082.stdout: parent: rbd/test1@snap2 2023-10-18T16:01:25.162 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info clone_v2 2023-10-18T16:01:25.162 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'op_features: clone-child' 2023-10-18T16:01:25.200 INFO:tasks.workunit.client.0.smithi082.stdout: op_features: clone-child 2023-10-18T16:01:25.201 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd export clone_v1 - 2023-10-18T16:01:25.201 INFO:tasks.workunit.client.0.smithi082.stderr:++ md5sum 2023-10-18T16:01:25.241 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 100% complete...done. 2023-10-18T16:01:25.244 INFO:tasks.workunit.client.0.smithi082.stderr:+ test '1addd36bf90e03ff605c772595892efd -' = '1addd36bf90e03ff605c772595892efd -' 2023-10-18T16:01:25.244 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd export clone_v2 - 2023-10-18T16:01:25.245 INFO:tasks.workunit.client.0.smithi082.stderr:++ md5sum 2023-10-18T16:01:25.283 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 100% complete...done. 2023-10-18T16:01:25.286 INFO:tasks.workunit.client.0.smithi082.stderr:+ test '1addd36bf90e03ff605c772595892efd -' = '1addd36bf90e03ff605c772595892efd -' 2023-10-18T16:01:25.287 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd children test1@snap1 2023-10-18T16:01:25.325 INFO:tasks.workunit.client.0.smithi082.stderr:+ test rbd/clone_v1 = rbd/clone_v1 2023-10-18T16:01:25.326 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd children test1@snap2 2023-10-18T16:01:25.360 INFO:tasks.workunit.client.0.smithi082.stderr:+ test rbd/clone_v2 = rbd/clone_v2 2023-10-18T16:01:25.360 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration prepare test1 rbd2/test2 2023-10-18T16:01:25.446 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:25.442+0000 7fcba375a700 0 -- 172.21.15.82:0/3297003471 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x5643fcf8a920 msgr2=0x5643fcfe6950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:26.933 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:26.929+0000 7fcba375a700 0 -- 172.21.15.82:0/3297003471 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7fcb88036a10 msgr2=0x7fcb88038e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:27.010 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info clone_v1 2023-10-18T16:01:27.011 INFO:tasks.workunit.client.0.smithi082.stderr:+ fgrep 'parent: rbd2/test2@snap1' 2023-10-18T16:01:27.061 INFO:tasks.workunit.client.0.smithi082.stdout: parent: rbd2/test2@snap1 2023-10-18T16:01:27.062 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info clone_v2 2023-10-18T16:01:27.062 INFO:tasks.workunit.client.0.smithi082.stderr:+ fgrep 'parent: rbd2/test2@snap2' 2023-10-18T16:01:27.114 INFO:tasks.workunit.client.0.smithi082.stdout: parent: rbd2/test2@snap2 2023-10-18T16:01:27.114 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info clone_v2 2023-10-18T16:01:27.114 INFO:tasks.workunit.client.0.smithi082.stderr:+ fgrep 'op_features: clone-child' 2023-10-18T16:01:27.166 INFO:tasks.workunit.client.0.smithi082.stdout: op_features: clone-child 2023-10-18T16:01:27.166 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd children rbd2/test2@snap1 2023-10-18T16:01:27.220 INFO:tasks.workunit.client.0.smithi082.stderr:+ test rbd/clone_v1 = rbd/clone_v1 2023-10-18T16:01:27.220 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd children rbd2/test2@snap2 2023-10-18T16:01:27.267 INFO:tasks.workunit.client.0.smithi082.stderr:+ test rbd/clone_v2 = rbd/clone_v2 2023-10-18T16:01:27.267 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration execute test1 2023-10-18T16:01:27.355 INFO:tasks.workunit.client.0.smithi082.stderr: Image migration: 100% complete...done. 2023-10-18T16:01:27.357 INFO:tasks.workunit.client.0.smithi082.stderr:+ expect_fail rbd migration commit test1 2023-10-18T16:01:27.358 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration commit test1 2023-10-18T16:01:27.423 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: the image has descendants: 2023-10-18T16:01:27.424 INFO:tasks.workunit.client.0.smithi082.stderr: rbd/clone_v1 2023-10-18T16:01:27.424 INFO:tasks.workunit.client.0.smithi082.stderr: rbd/clone_v2 2023-10-18T16:01:27.424 INFO:tasks.workunit.client.0.smithi082.stderr:Warning: in-use, read-only descendant images will not detect the parent update. 2023-10-18T16:01:27.424 INFO:tasks.workunit.client.0.smithi082.stderr:Ensure no descendant images are opened read-only and run again with force flag. 2023-10-18T16:01:27.426 INFO:tasks.workunit.client.0.smithi082.stderr:+ return 0 2023-10-18T16:01:27.426 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration commit test1 --force 2023-10-18T16:01:27.495 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: the image has descendants: 2023-10-18T16:01:27.495 INFO:tasks.workunit.client.0.smithi082.stderr: rbd/clone_v1 2023-10-18T16:01:27.495 INFO:tasks.workunit.client.0.smithi082.stderr: rbd/clone_v2 2023-10-18T16:01:27.496 INFO:tasks.workunit.client.0.smithi082.stderr:Warning: in-use, read-only descendant images will not detect the parent update. 2023-10-18T16:01:27.496 INFO:tasks.workunit.client.0.smithi082.stderr:Proceeding anyway due to force flag set. 2023-10-18T16:01:27.507 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:27.503+0000 7f961aac0700 0 -- 172.21.15.82:0/4029445249 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x55cdb1170800 msgr2=0x55cdb11c9a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:27.514 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:27.510+0000 7f961aac0700 0 -- 172.21.15.82:0/4029445249 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7f95fc036a60 msgr2=0x7f95fc038e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:28.929 INFO:tasks.workunit.client.0.smithi082.stderr: Commit image migration: 100% complete...done. 2023-10-18T16:01:28.932 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd export clone_v1 - 2023-10-18T16:01:28.933 INFO:tasks.workunit.client.0.smithi082.stderr:++ md5sum 2023-10-18T16:01:28.975 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 100% complete...done. 2023-10-18T16:01:28.979 INFO:tasks.workunit.client.0.smithi082.stderr:+ test '1addd36bf90e03ff605c772595892efd -' = '1addd36bf90e03ff605c772595892efd -' 2023-10-18T16:01:28.979 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd export clone_v2 - 2023-10-18T16:01:28.979 INFO:tasks.workunit.client.0.smithi082.stderr:++ md5sum 2023-10-18T16:01:29.019 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 100% complete...done. 2023-10-18T16:01:29.022 INFO:tasks.workunit.client.0.smithi082.stderr:+ test '1addd36bf90e03ff605c772595892efd -' = '1addd36bf90e03ff605c772595892efd -' 2023-10-18T16:01:29.023 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration prepare rbd2/test2 test1 2023-10-18T16:01:29.121 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:29.117+0000 7f8c80651700 0 -- 172.21.15.82:0/14143143 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x7f8c600046d0 msgr2=0x7f8c60004b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:30.941 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:30.937+0000 7f8c80e52700 0 -- 172.21.15.82:0/14143143 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x55f7888f2de0 msgr2=0x55f78894f4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:31.020 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info clone_v1 2023-10-18T16:01:31.020 INFO:tasks.workunit.client.0.smithi082.stderr:+ fgrep 'parent: rbd/test1@snap1' 2023-10-18T16:01:31.072 INFO:tasks.workunit.client.0.smithi082.stdout: parent: rbd/test1@snap1 2023-10-18T16:01:31.073 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info clone_v2 2023-10-18T16:01:31.073 INFO:tasks.workunit.client.0.smithi082.stderr:+ fgrep 'parent: rbd/test1@snap2' 2023-10-18T16:01:31.124 INFO:tasks.workunit.client.0.smithi082.stdout: parent: rbd/test1@snap2 2023-10-18T16:01:31.124 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info clone_v2 2023-10-18T16:01:31.124 INFO:tasks.workunit.client.0.smithi082.stderr:+ fgrep 'op_features: clone-child' 2023-10-18T16:01:31.175 INFO:tasks.workunit.client.0.smithi082.stdout: op_features: clone-child 2023-10-18T16:01:31.176 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd children test1@snap1 2023-10-18T16:01:31.228 INFO:tasks.workunit.client.0.smithi082.stderr:+ test rbd/clone_v1 = rbd/clone_v1 2023-10-18T16:01:31.228 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd children test1@snap2 2023-10-18T16:01:31.275 INFO:tasks.workunit.client.0.smithi082.stderr:+ test rbd/clone_v2 = rbd/clone_v2 2023-10-18T16:01:31.275 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration execute test1 2023-10-18T16:01:31.369 INFO:tasks.workunit.client.0.smithi082.stderr: Image migration: 100% complete...done. 2023-10-18T16:01:31.371 INFO:tasks.workunit.client.0.smithi082.stderr:+ expect_fail rbd migration commit test1 2023-10-18T16:01:31.372 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration commit test1 2023-10-18T16:01:31.447 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: the image has descendants: 2023-10-18T16:01:31.448 INFO:tasks.workunit.client.0.smithi082.stderr: rbd/clone_v1 2023-10-18T16:01:31.448 INFO:tasks.workunit.client.0.smithi082.stderr: rbd/clone_v2 2023-10-18T16:01:31.448 INFO:tasks.workunit.client.0.smithi082.stderr:Warning: in-use, read-only descendant images will not detect the parent update. 2023-10-18T16:01:31.448 INFO:tasks.workunit.client.0.smithi082.stderr:Ensure no descendant images are opened read-only and run again with force flag. 2023-10-18T16:01:31.450 INFO:tasks.workunit.client.0.smithi082.stderr:+ return 0 2023-10-18T16:01:31.450 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration commit test1 --force 2023-10-18T16:01:31.523 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: the image has descendants: 2023-10-18T16:01:31.523 INFO:tasks.workunit.client.0.smithi082.stderr: rbd/clone_v1 2023-10-18T16:01:31.523 INFO:tasks.workunit.client.0.smithi082.stderr: rbd/clone_v2 2023-10-18T16:01:31.524 INFO:tasks.workunit.client.0.smithi082.stderr:Warning: in-use, read-only descendant images will not detect the parent update. 2023-10-18T16:01:31.524 INFO:tasks.workunit.client.0.smithi082.stderr:Proceeding anyway due to force flag set. 2023-10-18T16:01:31.525 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:31.521+0000 7f50b7b35700 0 -- 172.21.15.82:0/2278171178 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x55706d721d40 msgr2=0x55706d77bbe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:31.534 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:31.531+0000 7f50b7334700 0 -- 172.21.15.82:0/2278171178 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x55706d721d40 msgr2=0x7f509c03e180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:32.973 INFO:tasks.workunit.client.0.smithi082.stderr: Commit image migration: 100% complete...done. 2023-10-18T16:01:32.977 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd export clone_v1 - 2023-10-18T16:01:32.978 INFO:tasks.workunit.client.0.smithi082.stderr:++ md5sum 2023-10-18T16:01:33.020 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 100% complete...done. 2023-10-18T16:01:33.025 INFO:tasks.workunit.client.0.smithi082.stderr:+ test '1addd36bf90e03ff605c772595892efd -' = '1addd36bf90e03ff605c772595892efd -' 2023-10-18T16:01:33.026 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd export clone_v2 - 2023-10-18T16:01:33.026 INFO:tasks.workunit.client.0.smithi082.stderr:++ md5sum 2023-10-18T16:01:33.065 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 100% complete...done. 2023-10-18T16:01:33.068 INFO:tasks.workunit.client.0.smithi082.stderr:+ test '1addd36bf90e03ff605c772595892efd -' = '1addd36bf90e03ff605c772595892efd -' 2023-10-18T16:01:33.068 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd remove clone_v1 2023-10-18T16:01:33.179 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:33.176+0000 7faf8b4f6700 0 -- 172.21.15.82:0/1551641623 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x558d4defa1c0 msgr2=0x558d4df1d180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:33.179 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:33.182 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd remove clone_v2 2023-10-18T16:01:33.286 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:33.284+0000 7f619806a700 0 -- 172.21.15.82:0/228013367 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x562812253520 msgr2=0x562812267cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:33.295 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:33.297 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap unprotect test1@snap1 2023-10-18T16:01:33.342 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap purge test1 2023-10-18T16:01:34.951 INFO:tasks.workunit.client.0.smithi082.stderr: Removing all snapshots: 50% complete... Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2023-10-18T16:01:34.963 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test1 2023-10-18T16:01:35.072 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:35.069+0000 7f462aa43700 0 -- 172.21.15.82:0/3248664905 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7f4608011dd0 msgr2=0x7f4608012210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:35.072 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:35.075 INFO:tasks.workunit.client.0.smithi082.stderr:+ for format in 1 2 2023-10-18T16:01:35.075 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create -s 128M --image-format 1 test2 2023-10-18T16:01:35.097 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: image format 1 is deprecated 2023-10-18T16:01:35.103 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:35.100+0000 7fb3901eb380 -1 librbd: Forced V1 image creation. 2023-10-18T16:01:35.111 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration prepare test2 --data-pool rbd2 2023-10-18T16:01:35.190 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-18T16:01:35.230 INFO:tasks.workunit.client.0.smithi082.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-18T16:01:35.236 INFO:tasks.workunit.client.0.smithi082.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-18T16:01:35.236 INFO:tasks.workunit.client.0.smithi082.stdout:elapsed: 0 ops: 1 ops/sec: 200.004 bytes/sec: 200 KiB/s 2023-10-18T16:01:35.242 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration abort test2 2023-10-18T16:01:35.332 INFO:tasks.workunit.client.0.smithi082.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete...2023-10-18T16:01:35.329+0000 7fe8db017700 0 -- 172.21.15.82:0/2431279296 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x55b9ed0cc320 msgr2=0x55b9ed1b8680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:35.343 INFO:tasks.workunit.client.0.smithi082.stderr: Abort image migration: 100% complete...done. 2023-10-18T16:01:35.346 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-18T16:01:35.381 INFO:tasks.workunit.client.0.smithi082.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-18T16:01:35.383 INFO:tasks.workunit.client.0.smithi082.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-18T16:01:35.383 INFO:tasks.workunit.client.0.smithi082.stdout:elapsed: 0 ops: 1 ops/sec: 500.01 bytes/sec: 500 KiB/s 2023-10-18T16:01:35.388 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test2 2023-10-18T16:01:35.435 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2023-10-18T16:01:35.437 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create -s 128M --image-format 1 test2 2023-10-18T16:01:35.459 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: image format 1 is deprecated 2023-10-18T16:01:35.465 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:35.462+0000 7f812e726380 -1 librbd: Forced V1 image creation. 2023-10-18T16:01:35.472 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration prepare test2 --data-pool rbd2 2023-10-18T16:01:35.542 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-18T16:01:35.584 INFO:tasks.workunit.client.0.smithi082.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-18T16:01:35.590 INFO:tasks.workunit.client.0.smithi082.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-18T16:01:35.590 INFO:tasks.workunit.client.0.smithi082.stdout:elapsed: 0 ops: 1 ops/sec: 166.67 bytes/sec: 167 KiB/s 2023-10-18T16:01:35.599 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration execute test2 2023-10-18T16:01:35.698 INFO:tasks.workunit.client.0.smithi082.stderr: Image migration: 3% complete... Image migration: 6% complete... Image migration: 9% complete... Image migration: 12% complete... Image migration: 15% complete... Image migration: 18% complete... Image migration: 21% complete... Image migration: 25% complete... Image migration: 28% complete... Image migration: 31% complete... Image migration: 34% complete... Image migration: 37% complete... Image migration: 40% complete... Image migration: 43% complete... Image migration: 46% complete... Image migration: 50% complete... Image migration: 53% complete... Image migration: 56% complete... Image migration: 59% complete... Image migration: 62% complete... Image migration: 65% complete... Image migration: 68% complete... Image migration: 71% complete... Image migration: 75% complete... Image migration: 78% complete... Image migration: 81% complete... Image migration: 84% complete... Image migration: 87% complete... Image migration: 90% complete... Image migration: 93% complete... Image migration: 96% complete...2023-10-18T16:01:35.695+0000 7f9312760700 0 -- 172.21.15.82:0/214383844 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7f92f8008780 msgr2=0x7f92f800abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-18T16:01:35.705 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:01:35.702+0000 7fa2ffef5700 -1 reset not still connected to 0x559c85cff930 2023-10-18T16:01:35.727 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:35.724+0000 7f9312f61700 0 -- 172.21.15.82:0/214383844 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x55bc92e80d30 msgr2=0x55bc92f6c900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:35.727 INFO:tasks.workunit.client.0.smithi082.stderr: Image migration: 100% complete...done. 2023-10-18T16:01:35.729 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration abort test2 2023-10-18T16:01:35.806 INFO:tasks.workunit.client.0.smithi082.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete...2023-10-18T16:01:35.803+0000 7f2b5a21a700 0 -- 172.21.15.82:0/1069699149 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x56120d8d92b0 msgr2=0x56120d9c5980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-18T16:01:35.824 INFO:tasks.workunit.client.0.smithi082.stderr: Abort image migration: 100% complete...done. 2023-10-18T16:01:35.826 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-18T16:01:35.861 INFO:tasks.workunit.client.0.smithi082.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-18T16:01:35.866 INFO:tasks.workunit.client.0.smithi082.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-18T16:01:35.866 INFO:tasks.workunit.client.0.smithi082.stdout:elapsed: 0 ops: 1 ops/sec: 200.004 bytes/sec: 200 KiB/s 2023-10-18T16:01:35.871 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test2 2023-10-18T16:01:35.930 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2023-10-18T16:01:35.932 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create -s 128M --image-format 1 test2 2023-10-18T16:01:35.954 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: image format 1 is deprecated 2023-10-18T16:01:35.960 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:35.957+0000 7f03987cf380 -1 librbd: Forced V1 image creation. 2023-10-18T16:01:35.968 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration prepare test2 --data-pool INVALID_DATA_POOL 2023-10-18T16:01:36.016 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:36.013+0000 7fa682656380 -1 librbd::image::CreateRequest: 0x55a13a90f690 validate_data_pool: data pool INVALID_DATA_POOL does not exist 2023-10-18T16:01:36.017 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:36.013+0000 7fa682656380 -1 librbd::Migration: create_dst_image: header creation failed: (2) No such file or directory 2023-10-18T16:01:36.024 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: preparing migration failed: (2) No such file or directory 2023-10-18T16:01:36.027 INFO:tasks.workunit.client.0.smithi082.stderr:+ true 2023-10-18T16:01:36.027 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-18T16:01:36.061 INFO:tasks.workunit.client.0.smithi082.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-18T16:01:36.063 INFO:tasks.workunit.client.0.smithi082.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-18T16:01:36.064 INFO:tasks.workunit.client.0.smithi082.stdout:elapsed: 0 ops: 1 ops/sec: 333.34 bytes/sec: 333 KiB/s 2023-10-18T16:01:36.068 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test2 2023-10-18T16:01:36.124 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2023-10-18T16:01:36.127 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create -s 128M --image-format 1 test2 2023-10-18T16:01:36.149 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: image format 1 is deprecated 2023-10-18T16:01:36.155 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:36.152+0000 7f40317cf380 -1 librbd: Forced V1 image creation. 2023-10-18T16:01:36.162 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration prepare test2 rbd2/test2 2023-10-18T16:01:36.224 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 rbd2/test2 2023-10-18T16:01:36.264 INFO:tasks.workunit.client.0.smithi082.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-18T16:01:36.268 INFO:tasks.workunit.client.0.smithi082.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-18T16:01:36.268 INFO:tasks.workunit.client.0.smithi082.stdout:elapsed: 0 ops: 1 ops/sec: 250.005 bytes/sec: 250 KiB/s 2023-10-18T16:01:36.276 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration abort test2 2023-10-18T16:01:36.351 INFO:tasks.workunit.client.0.smithi082.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete... Abort image migration: 100% complete...done. 2023-10-18T16:01:36.353 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-18T16:01:36.389 INFO:tasks.workunit.client.0.smithi082.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-18T16:01:36.392 INFO:tasks.workunit.client.0.smithi082.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-18T16:01:36.392 INFO:tasks.workunit.client.0.smithi082.stdout:elapsed: 0 ops: 1 ops/sec: 333.34 bytes/sec: 333 KiB/s 2023-10-18T16:01:36.398 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test2 2023-10-18T16:01:36.448 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2023-10-18T16:01:36.450 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create -s 128M --image-format 1 test2 2023-10-18T16:01:36.472 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: image format 1 is deprecated 2023-10-18T16:01:36.478 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:36.475+0000 7f36a7090380 -1 librbd: Forced V1 image creation. 2023-10-18T16:01:36.489 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration prepare test2 rbd2/test2 2023-10-18T16:01:36.560 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration abort rbd2/test2 2023-10-18T16:01:36.629 INFO:tasks.workunit.client.0.smithi082.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete...2023-10-18T16:01:36.627+0000 7f13e126c700 0 -- 172.21.15.82:0/2833365319 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x7f13bc008780 msgr2=0x7f13bc00abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:36.634 INFO:tasks.workunit.client.0.smithi082.stderr: Abort image migration: 100% complete...done. 2023-10-18T16:01:36.637 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-18T16:01:36.671 INFO:tasks.workunit.client.0.smithi082.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-18T16:01:36.674 INFO:tasks.workunit.client.0.smithi082.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-18T16:01:36.674 INFO:tasks.workunit.client.0.smithi082.stdout:elapsed: 0 ops: 1 ops/sec: 500.01 bytes/sec: 500 KiB/s 2023-10-18T16:01:36.679 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test2 2023-10-18T16:01:36.729 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2023-10-18T16:01:36.732 INFO:tasks.workunit.client.0.smithi082.stderr:+ test 1 = 1 2023-10-18T16:01:36.732 INFO:tasks.workunit.client.0.smithi082.stderr:+ continue 2023-10-18T16:01:36.732 INFO:tasks.workunit.client.0.smithi082.stderr:+ for format in 1 2 2023-10-18T16:01:36.732 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create -s 128M --image-format 2 test2 2023-10-18T16:01:36.776 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration prepare test2 --data-pool rbd2 2023-10-18T16:01:36.882 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:36.879+0000 7f90555f4700 0 -- 172.21.15.82:0/1210989189 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x5621a65a6340 msgr2=0x5621a6602870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:36.890 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-18T16:01:36.942 INFO:tasks.workunit.client.0.smithi082.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-18T16:01:36.947 INFO:tasks.workunit.client.0.smithi082.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-18T16:01:36.947 INFO:tasks.workunit.client.0.smithi082.stdout:elapsed: 0 ops: 1 ops/sec: 200.004 bytes/sec: 200 KiB/s 2023-10-18T16:01:36.954 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration abort test2 2023-10-18T16:01:37.057 INFO:tasks.workunit.client.0.smithi082.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete...2023-10-18T16:01:37.054+0000 7f0130bcb700 0 -- 172.21.15.82:0/428348590 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x55fa7eb8a320 msgr2=0x55fa7ec76840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:37.059 INFO:tasks.workunit.client.0.smithi082.stderr: Abort image migration: 100% complete...done. 2023-10-18T16:01:37.061 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-18T16:01:37.105 INFO:tasks.workunit.client.0.smithi082.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-18T16:01:37.108 INFO:tasks.workunit.client.0.smithi082.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-18T16:01:37.108 INFO:tasks.workunit.client.0.smithi082.stdout:elapsed: 0 ops: 1 ops/sec: 333.34 bytes/sec: 333 KiB/s 2023-10-18T16:01:37.119 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test2 2023-10-18T16:01:37.232 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete...2023-10-18T16:01:37.230+0000 7f031dd3c700 0 -- 172.21.15.82:0/4002864309 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x7f0300036c80 msgr2=0x7f03000390b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:37.245 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:37.248 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create -s 128M --image-format 2 test2 2023-10-18T16:01:37.490 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration prepare test2 --data-pool rbd2 2023-10-18T16:01:37.592 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:37.589+0000 7f58c0663700 0 -- 172.21.15.82:0/2417884579 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x7f58a4036c80 msgr2=0x7f58a40390b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:37.599 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-18T16:01:37.649 INFO:tasks.workunit.client.0.smithi082.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-18T16:01:37.657 INFO:tasks.workunit.client.0.smithi082.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-18T16:01:37.658 INFO:tasks.workunit.client.0.smithi082.stdout:elapsed: 0 ops: 1 ops/sec: 111.113 bytes/sec: 111 KiB/s 2023-10-18T16:01:37.669 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration execute test2 2023-10-18T16:01:37.769 INFO:tasks.workunit.client.0.smithi082.stderr: Image migration: 3% complete... Image migration: 6% complete... Image migration: 9% complete... Image migration: 12% complete... Image migration: 15% complete... Image migration: 18% complete... Image migration: 21% complete... Image migration: 25% complete... Image migration: 28% complete... Image migration: 31% complete... Image migration: 34% complete... Image migration: 37% complete... Image migration: 40% complete... Image migration: 43% complete... Image migration: 46% complete... Image migration: 50% complete... Image migration: 53% complete... Image migration: 56% complete... Image migration: 59% complete... Image migration: 62% complete... Image migration: 65% complete... Image migration: 68% complete... Image migration: 71% complete... Image migration: 75% complete... Image migration: 78% complete... Image migration: 81% complete... Image migration: 84% complete... Image migration: 87% complete... Image migration: 90% complete... Image migration: 93% complete... Image migration: 96% complete...2023-10-18T16:01:37.766+0000 7fe87fae6700 0 -- 172.21.15.82:0/3901126832 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x55f6e80a44c0 msgr2=0x55f6e80fe700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:37.802 INFO:tasks.workunit.client.0.smithi082.stderr: Image migration: 100% complete...done. 2023-10-18T16:01:37.804 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration abort test2 2023-10-18T16:01:37.908 INFO:tasks.workunit.client.0.smithi082.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete...2023-10-18T16:01:37.905+0000 7f38ed7d0700 0 -- 172.21.15.82:0/1195936680 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x7f38d0036c80 msgr2=0x7f38d00390b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:37.914 INFO:tasks.workunit.client.0.smithi082.stderr: Abort image migration: 100% complete...done. 2023-10-18T16:01:37.917 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-18T16:01:37.958 INFO:tasks.workunit.client.0.smithi082.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-18T16:01:37.960 INFO:tasks.workunit.client.0.smithi082.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-18T16:01:37.960 INFO:tasks.workunit.client.0.smithi082.stdout:elapsed: 0 ops: 1 ops/sec: 500.01 bytes/sec: 500 KiB/s 2023-10-18T16:01:37.970 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test2 2023-10-18T16:01:38.094 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete...2023-10-18T16:01:38.091+0000 7f00c37ae700 0 -- 172.21.15.82:0/3263891098 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7f00a0008780 msgr2=0x7f00a000abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:38.095 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:38.098 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create -s 128M --image-format 2 test2 2023-10-18T16:01:38.139 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration prepare test2 --data-pool INVALID_DATA_POOL 2023-10-18T16:01:38.205 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:38.202+0000 7f60b4a36380 -1 librbd::image::CreateRequest: 0x561379ad5690 validate_data_pool: data pool INVALID_DATA_POOL does not exist 2023-10-18T16:01:38.205 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:38.202+0000 7f60b4a36380 -1 librbd::Migration: create_dst_image: header creation failed: (2) No such file or directory 2023-10-18T16:01:38.225 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: preparing migration failed: (2) No such file or directory 2023-10-18T16:01:38.228 INFO:tasks.workunit.client.0.smithi082.stderr:+ true 2023-10-18T16:01:38.228 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-18T16:01:38.270 INFO:tasks.workunit.client.0.smithi082.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-18T16:01:38.274 INFO:tasks.workunit.client.0.smithi082.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-18T16:01:38.274 INFO:tasks.workunit.client.0.smithi082.stdout:elapsed: 0 ops: 1 ops/sec: 250.005 bytes/sec: 250 KiB/s 2023-10-18T16:01:38.286 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test2 2023-10-18T16:01:38.394 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete...2023-10-18T16:01:38.391+0000 7f29aaef0700 0 -- 172.21.15.82:0/3464244896 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7f2988014c40 msgr2=0x7f2988011db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:38.396 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:38.399 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create -s 128M --image-format 2 test2 2023-10-18T16:01:38.440 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration prepare test2 rbd2/test2 2023-10-18T16:01:38.546 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:38.543+0000 7fb60d1c4700 0 -- 172.21.15.82:0/2240029753 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x55b55ce8fb90 msgr2=0x55b55ced5e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-18T16:01:38.552 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 rbd2/test2 2023-10-18T16:01:38.598 INFO:tasks.workunit.client.0.smithi082.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-18T16:01:38.601 INFO:tasks.workunit.client.0.smithi082.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-18T16:01:38.601 INFO:tasks.workunit.client.0.smithi082.stdout:elapsed: 0 ops: 1 ops/sec: 333.34 bytes/sec: 333 KiB/s 2023-10-18T16:01:38.607 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration abort test2 2023-10-18T16:01:38.698 INFO:tasks.workunit.client.0.smithi082.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete... Abort image migration: 100% complete...done. 2023-10-18T16:01:38.701 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-18T16:01:38.746 INFO:tasks.workunit.client.0.smithi082.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-18T16:01:38.748 INFO:tasks.workunit.client.0.smithi082.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-18T16:01:38.748 INFO:tasks.workunit.client.0.smithi082.stdout:elapsed: 0 ops: 1 ops/sec: 500.01 bytes/sec: 500 KiB/s 2023-10-18T16:01:38.759 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test2 2023-10-18T16:01:38.869 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete...2023-10-18T16:01:38.866+0000 7fad7322e700 0 -- 172.21.15.82:0/104889694 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x55f6bad150a0 msgr2=0x55f6bacee9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:38.870 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:38.872 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create -s 128M --image-format 2 test2 2023-10-18T16:01:38.913 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration prepare test2 rbd2/test2 2023-10-18T16:01:39.013 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration abort rbd2/test2 2023-10-18T16:01:39.113 INFO:tasks.workunit.client.0.smithi082.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete... Abort image migration: 100% complete...done. 2023-10-18T16:01:39.116 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-18T16:01:39.157 INFO:tasks.workunit.client.0.smithi082.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-18T16:01:39.159 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:39.157+0000 7f7755dfe700 0 -- 172.21.15.82:0/2216289549 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x560112cfd230 msgr2=0x560112de62f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:39.161 INFO:tasks.workunit.client.0.smithi082.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-18T16:01:39.161 INFO:tasks.workunit.client.0.smithi082.stdout:elapsed: 0 ops: 1 ops/sec: 250.005 bytes/sec: 250 KiB/s 2023-10-18T16:01:39.169 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test2 2023-10-18T16:01:39.283 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete...2023-10-18T16:01:39.280+0000 7f950651d700 0 -- 172.21.15.82:0/231648548 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7f94ec008780 msgr2=0x7f94ec00abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:39.283 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:39.286 INFO:tasks.workunit.client.0.smithi082.stderr:+ test 2 = 1 2023-10-18T16:01:39.286 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create -s 128M --image-format 2 test2 2023-10-18T16:01:39.328 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration prepare test2 rbd2/ns1/test3 2023-10-18T16:01:39.420 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:39.417+0000 7f85a6937700 0 -- 172.21.15.82:0/1879506850 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x55829a0051e0 msgr2=0x55829a0075d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:39.427 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 rbd2/ns1/test3 2023-10-18T16:01:39.473 INFO:tasks.workunit.client.0.smithi082.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-18T16:01:39.474 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:39.471+0000 7f1a39715700 0 -- 172.21.15.82:0/1093010630 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x556845073650 msgr2=0x556845086ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:39.478 INFO:tasks.workunit.client.0.smithi082.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-18T16:01:39.478 INFO:tasks.workunit.client.0.smithi082.stdout:elapsed: 0 ops: 1 ops/sec: 200.004 bytes/sec: 200 KiB/s 2023-10-18T16:01:39.485 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd migration abort test2 2023-10-18T16:01:39.594 INFO:tasks.workunit.client.0.smithi082.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete... Abort image migration: 100% complete...done. 2023-10-18T16:01:39.597 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-18T16:01:39.639 INFO:tasks.workunit.client.0.smithi082.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-18T16:01:39.641 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:39.638+0000 7f4d79715700 0 -- 172.21.15.82:0/356409956 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7f4d54015c00 msgr2=0x7f4d540120f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:39.641 INFO:tasks.workunit.client.0.smithi082.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-18T16:01:39.642 INFO:tasks.workunit.client.0.smithi082.stdout:elapsed: 0 ops: 1 ops/sec: 500.01 bytes/sec: 500 KiB/s 2023-10-18T16:01:39.652 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test2 2023-10-18T16:01:39.760 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete...2023-10-18T16:01:39.757+0000 7f9b6b3ed700 0 -- 172.21.15.82:0/3489740927 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7f9b4c006040 msgr2=0x7f9b4c008470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:39.760 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:39.763 INFO:tasks.workunit.client.0.smithi082.stderr:+ remove_images 2023-10-18T16:01:39.764 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:39.837 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:39.917 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:39.995 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:40.072 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:40.151 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:40.229 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:40.303 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:40.378 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:40.452 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:40.529 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:40.610 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:40.689 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:40.766 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:40.839 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:40.911 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:40.985 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:41.060 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:41.135 INFO:tasks.workunit.client.0.smithi082.stderr:+ ceph osd pool rm rbd2 rbd2 --yes-i-really-really-mean-it 2023-10-18T16:01:42.062 INFO:tasks.workunit.client.0.smithi082.stderr:pool 'rbd2' does not exist 2023-10-18T16:01:42.072 INFO:tasks.workunit.client.0.smithi082.stderr:+ test_config 2023-10-18T16:01:42.072 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'testing config...' 2023-10-18T16:01:42.073 INFO:tasks.workunit.client.0.smithi082.stdout:testing config... 2023-10-18T16:01:42.073 INFO:tasks.workunit.client.0.smithi082.stderr:+ remove_images 2023-10-18T16:01:42.073 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:42.152 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:42.225 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:42.302 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:42.380 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:42.457 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:42.530 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:42.603 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:42.676 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:42.751 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:42.826 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:42.901 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:42.976 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:43.048 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:43.124 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:43.199 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:43.278 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:43.352 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:43.425 INFO:tasks.workunit.client.0.smithi082.stderr:+ expect_fail rbd config global set osd rbd_cache true 2023-10-18T16:01:43.426 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config global set osd rbd_cache true 2023-10-18T16:01:43.447 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: invalid config entity: osd (must be global, client or client.) 2023-10-18T16:01:43.449 INFO:tasks.workunit.client.0.smithi082.stderr:+ return 0 2023-10-18T16:01:43.449 INFO:tasks.workunit.client.0.smithi082.stderr:+ expect_fail rbd config global set global debug_ms 10 2023-10-18T16:01:43.449 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config global set global debug_ms 10 2023-10-18T16:01:43.471 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: not rbd option: debug_ms 2023-10-18T16:01:43.472 INFO:tasks.workunit.client.0.smithi082.stderr:+ return 0 2023-10-18T16:01:43.473 INFO:tasks.workunit.client.0.smithi082.stderr:+ expect_fail rbd config global set global rbd_UNKNOWN false 2023-10-18T16:01:43.473 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config global set global rbd_UNKNOWN false 2023-10-18T16:01:43.494 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: invalid config key: rbd_UNKNOWN 2023-10-18T16:01:43.496 INFO:tasks.workunit.client.0.smithi082.stderr:+ return 0 2023-10-18T16:01:43.496 INFO:tasks.workunit.client.0.smithi082.stderr:+ expect_fail rbd config global set global rbd_cache INVALID 2023-10-18T16:01:43.496 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config global set global rbd_cache INVALID 2023-10-18T16:01:43.523 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: error setting rbd_cache: error parsing value: Expected option value to be integer, got 'INVALID' 2023-10-18T16:01:43.525 INFO:tasks.workunit.client.0.smithi082.stderr:+ return 0 2023-10-18T16:01:43.525 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config global set global rbd_cache false 2023-10-18T16:01:43.557 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config global set client rbd_cache true 2023-10-18T16:01:43.588 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config global set client.123 rbd_cache false 2023-10-18T16:01:43.619 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config global get global rbd_cache 2023-10-18T16:01:43.620 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^false$' 2023-10-18T16:01:43.647 INFO:tasks.workunit.client.0.smithi082.stdout:false 2023-10-18T16:01:43.647 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config global get client rbd_cache 2023-10-18T16:01:43.648 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^true$' 2023-10-18T16:01:43.675 INFO:tasks.workunit.client.0.smithi082.stdout:true 2023-10-18T16:01:43.676 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config global get client.123 rbd_cache 2023-10-18T16:01:43.676 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^false$' 2023-10-18T16:01:43.703 INFO:tasks.workunit.client.0.smithi082.stdout:false 2023-10-18T16:01:43.704 INFO:tasks.workunit.client.0.smithi082.stderr:+ expect_fail rbd config global get client.UNKNOWN rbd_cache 2023-10-18T16:01:43.704 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config global get client.UNKNOWN rbd_cache 2023-10-18T16:01:43.730 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: rbd_cache is not set 2023-10-18T16:01:43.732 INFO:tasks.workunit.client.0.smithi082.stderr:+ return 0 2023-10-18T16:01:43.732 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config global list global 2023-10-18T16:01:43.733 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^rbd_cache * false * global *$' 2023-10-18T16:01:43.759 INFO:tasks.workunit.client.0.smithi082.stdout:rbd_cache false global 2023-10-18T16:01:43.760 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config global list client 2023-10-18T16:01:43.760 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^rbd_cache * true * client *$' 2023-10-18T16:01:43.786 INFO:tasks.workunit.client.0.smithi082.stdout:rbd_cache true client 2023-10-18T16:01:43.787 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config global list client.123 2023-10-18T16:01:43.787 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^rbd_cache * false * client.123 *$' 2023-10-18T16:01:43.813 INFO:tasks.workunit.client.0.smithi082.stdout:rbd_cache false client.123 2023-10-18T16:01:43.814 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config global list client.UNKNOWN 2023-10-18T16:01:43.814 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^rbd_cache * true * client *$' 2023-10-18T16:01:43.840 INFO:tasks.workunit.client.0.smithi082.stdout:rbd_cache true client 2023-10-18T16:01:43.841 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config global rm client rbd_cache 2023-10-18T16:01:43.871 INFO:tasks.workunit.client.0.smithi082.stderr:+ expect_fail rbd config global get client rbd_cache 2023-10-18T16:01:43.871 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config global get client rbd_cache 2023-10-18T16:01:43.896 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: rbd_cache is not set 2023-10-18T16:01:43.898 INFO:tasks.workunit.client.0.smithi082.stderr:+ return 0 2023-10-18T16:01:43.899 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config global list client 2023-10-18T16:01:43.899 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^rbd_cache * false * global *$' 2023-10-18T16:01:43.925 INFO:tasks.workunit.client.0.smithi082.stdout:rbd_cache false global 2023-10-18T16:01:43.926 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config global rm client.123 rbd_cache 2023-10-18T16:01:43.955 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config global rm global rbd_cache 2023-10-18T16:01:43.986 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config pool set rbd rbd_cache true 2023-10-18T16:01:44.022 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config pool list rbd 2023-10-18T16:01:44.022 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^rbd_cache * true * pool *$' 2023-10-18T16:01:44.050 INFO:tasks.workunit.client.0.smithi082.stdout:rbd_cache true pool 2023-10-18T16:01:44.051 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config pool get rbd rbd_cache 2023-10-18T16:01:44.051 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^true$' 2023-10-18T16:01:44.079 INFO:tasks.workunit.client.0.smithi082.stdout:true 2023-10-18T16:01:44.079 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create -s 1 test1 2023-10-18T16:01:44.105 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:44.102+0000 7fb2571f1380 -1 librbd: Forced V1 image creation. 2023-10-18T16:01:44.112 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config image list rbd/test1 2023-10-18T16:01:44.112 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^rbd_cache * true * pool *$' 2023-10-18T16:01:44.149 INFO:tasks.workunit.client.0.smithi082.stdout:rbd_cache true pool 2023-10-18T16:01:44.149 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config image set rbd/test1 rbd_cache false 2023-10-18T16:01:44.190 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config image list rbd/test1 2023-10-18T16:01:44.191 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^rbd_cache * false * image *$' 2023-10-18T16:01:44.427 INFO:tasks.workunit.client.0.smithi082.stdout:rbd_cache false image 2023-10-18T16:01:44.428 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config image get rbd/test1 rbd_cache 2023-10-18T16:01:44.428 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^false$' 2023-10-18T16:01:44.664 INFO:tasks.workunit.client.0.smithi082.stdout:false 2023-10-18T16:01:44.664 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config image remove rbd/test1 rbd_cache 2023-10-18T16:01:44.703 INFO:tasks.workunit.client.0.smithi082.stderr:+ expect_fail rbd config image get rbd/test1 rbd_cache 2023-10-18T16:01:44.703 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config image get rbd/test1 rbd_cache 2023-10-18T16:01:44.734 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: rbd_cache is not set 2023-10-18T16:01:44.739 INFO:tasks.workunit.client.0.smithi082.stderr:+ return 0 2023-10-18T16:01:44.739 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config image list rbd/test1 2023-10-18T16:01:44.740 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^rbd_cache * true * pool *$' 2023-10-18T16:01:44.774 INFO:tasks.workunit.client.0.smithi082.stdout:rbd_cache true pool 2023-10-18T16:01:44.775 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config pool remove rbd rbd_cache 2023-10-18T16:01:44.805 INFO:tasks.workunit.client.0.smithi082.stderr:+ expect_fail rbd config pool get rbd rbd_cache 2023-10-18T16:01:44.806 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config pool get rbd rbd_cache 2023-10-18T16:01:44.832 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: rbd_cache is not set 2023-10-18T16:01:44.833 INFO:tasks.workunit.client.0.smithi082.stderr:+ return 0 2023-10-18T16:01:44.834 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd config pool list rbd 2023-10-18T16:01:44.834 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^rbd_cache * true * config *$' 2023-10-18T16:01:44.861 INFO:tasks.workunit.client.0.smithi082.stdout:rbd_cache true config 2023-10-18T16:01:44.862 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test1 2023-10-18T16:01:44.898 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:44.901 INFO:tasks.workunit.client.0.smithi082.stderr:+ RBD_CREATE_ARGS= 2023-10-18T16:01:44.901 INFO:tasks.workunit.client.0.smithi082.stderr:+ test_others 2023-10-18T16:01:44.901 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'testing import, export, resize, and snapshots...' 2023-10-18T16:01:44.902 INFO:tasks.workunit.client.0.smithi082.stdout:testing import, export, resize, and snapshots... 2023-10-18T16:01:44.902 INFO:tasks.workunit.client.0.smithi082.stderr:+ TMP_FILES='/tmp/img1 /tmp/img1.new /tmp/img2 /tmp/img2.new /tmp/img3 /tmp/img3.new /tmp/img-diff1.new /tmp/img-diff2.new /tmp/img-diff3.new /tmp/img1.snap1 /tmp/img1.snap1 /tmp/img-diff1.snap1' 2023-10-18T16:01:44.903 INFO:tasks.workunit.client.0.smithi082.stderr:+ remove_images 2023-10-18T16:01:44.903 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:44.973 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:45.050 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:45.122 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:45.195 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:45.270 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:45.346 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:45.421 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:45.495 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:45.568 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:45.640 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:45.711 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:45.785 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:45.861 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:45.937 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:46.014 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:46.088 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:46.161 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:46.235 INFO:tasks.workunit.client.0.smithi082.stderr:+ rm -f /tmp/img1 /tmp/img1.new /tmp/img2 /tmp/img2.new /tmp/img3 /tmp/img3.new /tmp/img-diff1.new /tmp/img-diff2.new /tmp/img-diff3.new /tmp/img1.snap1 /tmp/img1.snap1 /tmp/img-diff1.snap1 2023-10-18T16:01:46.236 INFO:tasks.workunit.client.0.smithi082.stderr:+ dd if=/bin/sh of=/tmp/img1 bs=1k count=1 seek=10 2023-10-18T16:01:46.236 INFO:tasks.workunit.client.0.smithi082.stderr:1+0 records in 2023-10-18T16:01:46.237 INFO:tasks.workunit.client.0.smithi082.stderr:1+0 records out 2023-10-18T16:01:46.237 INFO:tasks.workunit.client.0.smithi082.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 6.3208e-05 s, 16.2 MB/s 2023-10-18T16:01:46.237 INFO:tasks.workunit.client.0.smithi082.stderr:+ dd if=/bin/dd of=/tmp/img1 bs=1k count=10 seek=100 2023-10-18T16:01:46.238 INFO:tasks.workunit.client.0.smithi082.stderr:10+0 records in 2023-10-18T16:01:46.238 INFO:tasks.workunit.client.0.smithi082.stderr:10+0 records out 2023-10-18T16:01:46.238 INFO:tasks.workunit.client.0.smithi082.stderr:10240 bytes (10 kB, 10 KiB) copied, 4.977e-05 s, 206 MB/s 2023-10-18T16:01:46.239 INFO:tasks.workunit.client.0.smithi082.stderr:+ dd if=/bin/rm of=/tmp/img1 bs=1k count=100 seek=1000 2023-10-18T16:01:46.239 INFO:tasks.workunit.client.0.smithi082.stderr:70+1 records in 2023-10-18T16:01:46.239 INFO:tasks.workunit.client.0.smithi082.stderr:70+1 records out 2023-10-18T16:01:46.239 INFO:tasks.workunit.client.0.smithi082.stderr:72216 bytes (72 kB, 71 KiB) copied, 0.000166373 s, 434 MB/s 2023-10-18T16:01:46.239 INFO:tasks.workunit.client.0.smithi082.stderr:+ dd if=/bin/ls of=/tmp/img1 bs=1k seek=10000 2023-10-18T16:01:46.240 INFO:tasks.workunit.client.0.smithi082.stderr:140+1 records in 2023-10-18T16:01:46.240 INFO:tasks.workunit.client.0.smithi082.stderr:140+1 records out 2023-10-18T16:01:46.240 INFO:tasks.workunit.client.0.smithi082.stderr:143368 bytes (143 kB, 140 KiB) copied, 0.000320149 s, 448 MB/s 2023-10-18T16:01:46.241 INFO:tasks.workunit.client.0.smithi082.stderr:+ dd if=/bin/ln of=/tmp/img1 bs=1k seek=100000 2023-10-18T16:01:46.241 INFO:tasks.workunit.client.0.smithi082.stderr:70+1 records in 2023-10-18T16:01:46.241 INFO:tasks.workunit.client.0.smithi082.stderr:70+1 records out 2023-10-18T16:01:46.241 INFO:tasks.workunit.client.0.smithi082.stderr:72320 bytes (72 kB, 71 KiB) copied, 0.000187692 s, 385 MB/s 2023-10-18T16:01:46.241 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd import /tmp/img1 testimg1 2023-10-18T16:01:46.265 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:46.262+0000 7f3b719a1380 -1 librbd: Forced V1 image creation. 2023-10-18T16:01:46.340 INFO:tasks.workunit.client.0.smithi082.stderr: Importing image: 4% complete... Importing image: 8% complete... Importing image: 12% complete... Importing image: 16% complete... Importing image: 20% complete... Importing image: 24% complete... Importing image: 28% complete... Importing image: 32% complete... Importing image: 36% complete... Importing image: 40% complete... Importing image: 45% complete... Importing image: 49% complete... Importing image: 53% complete... Importing image: 57% complete... Importing image: 61% complete... Importing image: 65% complete... Importing image: 69% complete... Importing image: 73% complete... Importing image: 77% complete... Importing image: 81% complete... Importing image: 85% complete... Importing image: 90% complete... Importing image: 94% complete... Importing image: 98% complete... Importing image: 100% complete...done. 2023-10-18T16:01:46.343 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd resize testimg1 --size=256 --allow-shrink 2023-10-18T16:01:46.378 INFO:tasks.workunit.client.0.smithi082.stderr: Resizing image: 100% complete...done. 2023-10-18T16:01:46.381 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd export testimg1 /tmp/img2 2023-10-18T16:01:46.468 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-10-18T16:01:46.472 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create testimg1 --snap=snap1 2023-10-18T16:01:47.026 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:01:47.031 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd resize testimg1 --size=128 2023-10-18T16:01:47.064 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: shrinking an image is only allowed with the --allow-shrink flag 2023-10-18T16:01:47.068 INFO:tasks.workunit.client.0.smithi082.stderr:+ true 2023-10-18T16:01:47.068 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd resize testimg1 --size=128 --allow-shrink 2023-10-18T16:01:47.114 INFO:tasks.workunit.client.0.smithi082.stderr: Resizing image: 50% complete... Resizing image: 51% complete... Resizing image: 53% complete... Resizing image: 54% complete... Resizing image: 56% complete... Resizing image: 57% complete... Resizing image: 59% complete... Resizing image: 60% complete... Resizing image: 62% complete... Resizing image: 64% complete... Resizing image: 65% complete... Resizing image: 67% complete... Resizing image: 68% complete... Resizing image: 70% complete... Resizing image: 71% complete... Resizing image: 73% complete... Resizing image: 75% complete... Resizing image: 76% complete... Resizing image: 78% complete... Resizing image: 79% complete... Resizing image: 81% complete... Resizing image: 82% complete... Resizing image: 84% complete... Resizing image: 85% complete... Resizing image: 87% complete... Resizing image: 89% complete... Resizing image: 90% complete... Resizing image: 92% complete... Resizing image: 93% complete... Resizing image: 95% complete... Resizing image: 96% complete... Resizing image: 98% complete... Resizing image: 100% complete...done. 2023-10-18T16:01:47.118 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd export testimg1 /tmp/img3 2023-10-18T16:01:47.189 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-10-18T16:01:47.192 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg1 2023-10-18T16:01:47.192 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 128 MiB' 2023-10-18T16:01:47.223 INFO:tasks.workunit.client.0.smithi082.stdout: size 128 MiB in 32 objects 2023-10-18T16:01:47.223 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info --snap=snap1 testimg1 2023-10-18T16:01:47.223 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 256 MiB' 2023-10-18T16:01:47.254 INFO:tasks.workunit.client.0.smithi082.stdout: size 256 MiB in 64 objects 2023-10-18T16:01:47.255 INFO:tasks.workunit.client.0.smithi082.stderr:+ rm -rf /tmp/diff-testimg1-1 /tmp/diff-testimg1-2 2023-10-18T16:01:47.255 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd export-diff testimg1 --snap=snap1 /tmp/diff-testimg1-1 2023-10-18T16:01:47.346 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 3% complete... Exporting image: 37% complete... Exporting image: 100% complete...done. 2023-10-18T16:01:47.352 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd export-diff testimg1 --from-snap=snap1 /tmp/diff-testimg1-2 2023-10-18T16:01:47.396 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 100% complete...done. 2023-10-18T16:01:47.400 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --size=1 testimg-diff1 2023-10-18T16:01:47.426 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:47.424+0000 7f476d641380 -1 librbd: Forced V1 image creation. 2023-10-18T16:01:47.433 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd import-diff --sparse-size 8K /tmp/diff-testimg1-1 testimg-diff1 2023-10-18T16:01:48.031 INFO:tasks.workunit.client.0.smithi082.stderr: Importing image diff: 22% complete... Importing image diff: 63% complete... Importing image diff: 99% complete... Importing image diff: 100% complete...done. 2023-10-18T16:01:48.040 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd import-diff --sparse-size 8K /tmp/diff-testimg1-2 testimg-diff1 2023-10-18T16:01:48.098 INFO:tasks.workunit.client.0.smithi082.stderr: Importing image diff: 68% complete... Importing image diff: 96% complete... Importing image diff: 100% complete...done. 2023-10-18T16:01:48.103 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg1 2023-10-18T16:01:48.104 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 128 MiB' 2023-10-18T16:01:48.135 INFO:tasks.workunit.client.0.smithi082.stdout: size 128 MiB in 32 objects 2023-10-18T16:01:48.136 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info --snap=snap1 testimg1 2023-10-18T16:01:48.136 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 256 MiB' 2023-10-18T16:01:48.166 INFO:tasks.workunit.client.0.smithi082.stdout: size 256 MiB in 64 objects 2023-10-18T16:01:48.167 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg-diff1 2023-10-18T16:01:48.167 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 128 MiB' 2023-10-18T16:01:48.197 INFO:tasks.workunit.client.0.smithi082.stdout: size 128 MiB in 32 objects 2023-10-18T16:01:48.198 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info --snap=snap1 testimg-diff1 2023-10-18T16:01:48.198 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 256 MiB' 2023-10-18T16:01:48.228 INFO:tasks.workunit.client.0.smithi082.stdout: size 256 MiB in 64 objects 2023-10-18T16:01:48.229 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd copy testimg1 --snap=snap1 testimg2 2023-10-18T16:01:48.323 INFO:tasks.workunit.client.0.smithi082.stderr: Image copy: 1% complete... Image copy: 3% complete... Image copy: 4% complete... Image copy: 6% complete... Image copy: 7% complete... Image copy: 9% complete... Image copy: 10% complete... Image copy: 12% complete... Image copy: 14% complete... Image copy: 15% complete... Image copy: 17% complete... Image copy: 18% complete... Image copy: 20% complete... Image copy: 21% complete... Image copy: 23% complete... Image copy: 25% complete... Image copy: 26% complete... Image copy: 28% complete... Image copy: 29% complete... Image copy: 31% complete... Image copy: 32% complete... Image copy: 34% complete... Image copy: 35% complete... Image copy: 37% complete... Image copy: 39% complete... Image copy: 40% complete... Image copy: 42% complete... Image copy: 43% complete... Image copy: 45% complete... Image copy: 46% complete... Image copy: 48% complete... Image copy: 50% complete... Image copy: 51% complete... Image copy: 53% complete... Image copy: 54% complete... Image copy: 56% complete... Image copy: 57% complete... Image copy: 59% complete... Image copy: 60% complete... Image copy: 62% complete... Image copy: 64% complete... Image copy: 65% complete... Image copy: 67% complete... Image copy: 68% complete... Image copy: 70% complete... Image copy: 71% complete... Image copy: 73% complete... Image copy: 75% complete... Image copy: 76% complete... Image copy: 78% complete... Image copy: 79% complete... Image copy: 81% complete... Image copy: 82% complete... Image copy: 84% complete... Image copy: 85% complete... Image copy: 87% complete... Image copy: 89% complete... Image copy: 90% complete... Image copy: 92% complete... Image copy: 93% complete... Image copy: 95% complete... Image copy: 96% complete... Image copy: 98% complete... Image copy: 100% complete... Image copy: 100% complete...done. 2023-10-18T16:01:48.326 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd copy testimg1 testimg3 2023-10-18T16:01:48.403 INFO:tasks.workunit.client.0.smithi082.stderr: Image copy: 3% complete... Image copy: 6% complete... Image copy: 9% complete... Image copy: 12% complete... Image copy: 15% complete... Image copy: 18% complete... Image copy: 21% complete... Image copy: 25% complete... Image copy: 28% complete... Image copy: 31% complete... Image copy: 34% complete... Image copy: 37% complete... Image copy: 40% complete... Image copy: 43% complete... Image copy: 46% complete... Image copy: 50% complete... Image copy: 53% complete... Image copy: 56% complete... Image copy: 59% complete... Image copy: 62% complete... Image copy: 65% complete... Image copy: 68% complete... Image copy: 71% complete... Image copy: 75% complete... Image copy: 78% complete... Image copy: 81% complete... Image copy: 84% complete... Image copy: 87% complete... Image copy: 90% complete... Image copy: 93% complete... Image copy: 96% complete... Image copy: 100% complete... Image copy: 100% complete...done. 2023-10-18T16:01:48.406 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd copy testimg-diff1 --sparse-size 768K --snap=snap1 testimg-diff2 2023-10-18T16:01:48.508 INFO:tasks.workunit.client.0.smithi082.stderr: Image copy: 1% complete... Image copy: 3% complete... Image copy: 4% complete... Image copy: 6% complete... Image copy: 7% complete... Image copy: 9% complete... Image copy: 10% complete... Image copy: 12% complete... Image copy: 14% complete... Image copy: 15% complete... Image copy: 17% complete... Image copy: 18% complete... Image copy: 20% complete... Image copy: 21% complete... Image copy: 23% complete... Image copy: 25% complete... Image copy: 26% complete... Image copy: 28% complete... Image copy: 29% complete... Image copy: 31% complete... Image copy: 32% complete... Image copy: 34% complete... Image copy: 35% complete... Image copy: 37% complete... Image copy: 39% complete... Image copy: 40% complete... Image copy: 42% complete... Image copy: 43% complete... Image copy: 45% complete... Image copy: 46% complete... Image copy: 48% complete... Image copy: 50% complete... Image copy: 51% complete... Image copy: 53% complete... Image copy: 54% complete... Image copy: 56% complete... Image copy: 57% complete... Image copy: 59% complete... Image copy: 60% complete... Image copy: 62% complete... Image copy: 64% complete... Image copy: 65% complete... Image copy: 67% complete... Image copy: 68% complete... Image copy: 70% complete... Image copy: 71% complete... Image copy: 73% complete... Image copy: 75% complete... Image copy: 76% complete... Image copy: 78% complete... Image copy: 79% complete... Image copy: 81% complete... Image copy: 82% complete... Image copy: 84% complete... Image copy: 85% complete... Image copy: 87% complete... Image copy: 89% complete... Image copy: 90% complete... Image copy: 92% complete... Image copy: 93% complete... Image copy: 95% complete... Image copy: 96% complete... Image copy: 98% complete... Image copy: 100% complete... Image copy: 100% complete...done. 2023-10-18T16:01:48.511 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd copy testimg-diff1 --sparse-size 768K testimg-diff3 2023-10-18T16:01:48.606 INFO:tasks.workunit.client.0.smithi082.stderr: Image copy: 3% complete... Image copy: 6% complete... Image copy: 9% complete... Image copy: 12% complete... Image copy: 15% complete... Image copy: 18% complete... Image copy: 21% complete... Image copy: 25% complete... Image copy: 28% complete... Image copy: 31% complete... Image copy: 34% complete... Image copy: 37% complete... Image copy: 40% complete... Image copy: 43% complete... Image copy: 46% complete... Image copy: 50% complete... Image copy: 53% complete... Image copy: 56% complete... Image copy: 59% complete... Image copy: 62% complete... Image copy: 65% complete... Image copy: 68% complete... Image copy: 71% complete... Image copy: 75% complete... Image copy: 78% complete... Image copy: 81% complete... Image copy: 84% complete... Image copy: 87% complete... Image copy: 90% complete... Image copy: 93% complete... Image copy: 96% complete... Image copy: 100% complete... Image copy: 100% complete...done. 2023-10-18T16:01:48.612 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg2 2023-10-18T16:01:48.613 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 256 MiB' 2023-10-18T16:01:48.644 INFO:tasks.workunit.client.0.smithi082.stdout: size 256 MiB in 64 objects 2023-10-18T16:01:48.645 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg3 2023-10-18T16:01:48.645 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 128 MiB' 2023-10-18T16:01:48.676 INFO:tasks.workunit.client.0.smithi082.stdout: size 128 MiB in 32 objects 2023-10-18T16:01:48.677 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg-diff2 2023-10-18T16:01:48.677 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 256 MiB' 2023-10-18T16:01:48.709 INFO:tasks.workunit.client.0.smithi082.stdout: size 256 MiB in 64 objects 2023-10-18T16:01:48.710 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg-diff3 2023-10-18T16:01:48.710 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 128 MiB' 2023-10-18T16:01:48.742 INFO:tasks.workunit.client.0.smithi082.stdout: size 128 MiB in 32 objects 2023-10-18T16:01:48.742 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd deep copy testimg1 testimg4 2023-10-18T16:01:49.048 INFO:tasks.workunit.client.0.smithi082.stderr: Image deep copy: 1% complete... Image deep copy: 3% complete...2023-10-18T16:01:49.045+0000 7fa6e575e700 0 -- 172.21.15.82:0/3225617005 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x7fa6c4001c20 msgr2=0x7fa6c803b4b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:49.069 INFO:tasks.workunit.client.0.smithi082.stderr: Image deep copy: 4% complete... Image deep copy: 6% complete... Image deep copy: 7% complete... Image deep copy: 9% complete... Image deep copy: 10% complete... Image deep copy: 12% complete... Image deep copy: 14% complete... Image deep copy: 15% complete... Image deep copy: 17% complete... Image deep copy: 18% complete... Image deep copy: 20% complete... Image deep copy: 21% complete... Image deep copy: 23% complete... Image deep copy: 25% complete... Image deep copy: 26% complete... Image deep copy: 28% complete... Image deep copy: 29% complete... Image deep copy: 31% complete... Image deep copy: 32% complete... Image deep copy: 34% complete... Image deep copy: 35% complete... Image deep copy: 37% complete... Image deep copy: 39% complete... Image deep copy: 40% complete... Image deep copy: 42% complete... Image deep copy: 43% complete... Image deep copy: 45% complete... Image deep copy: 46% complete... Image deep copy: 48% complete... Image deep copy: 50% complete... Image deep copy: 51% complete... Image deep copy: 53% complete... Image deep copy: 54% complete... Image deep copy: 56% complete... Image deep copy: 57% complete... Image deep copy: 59% complete... Image deep copy: 60% complete... Image deep copy: 62% complete... Image deep copy: 64% complete... Image deep copy: 65% complete... Image deep copy: 67% complete... Image deep copy: 68% complete... Image deep copy: 70% complete... Image deep copy: 71% complete... Image deep copy: 73% complete... Image deep copy: 75% complete... Image deep copy: 76% complete... Image deep copy: 78% complete... Image deep copy: 79% complete... Image deep copy: 81% complete... Image deep copy: 82% complete... Image deep copy: 84% complete... Image deep copy: 85% complete... Image deep copy: 87% complete... Image deep copy: 89% complete... Image deep copy: 90% complete... Image deep copy: 92% complete... Image deep copy: 93% complete... Image deep copy: 95% complete... Image deep copy: 96% complete... Image deep copy: 98% complete... Image deep copy: 100% complete... Image deep copy: 100% complete...done. 2023-10-18T16:01:49.072 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd deep copy testimg1 --snap=snap1 testimg5 2023-10-18T16:01:50.090 INFO:tasks.workunit.client.0.smithi082.stderr: Image deep copy: 1% complete... Image deep copy: 3% complete... Image deep copy: 4% complete... Image deep copy: 6% complete... Image deep copy: 7% complete... Image deep copy: 9% complete... Image deep copy: 10% complete... Image deep copy: 12% complete... Image deep copy: 14% complete... Image deep copy: 15% complete... Image deep copy: 17% complete... Image deep copy: 18% complete... Image deep copy: 20% complete... Image deep copy: 21% complete... Image deep copy: 23% complete... Image deep copy: 25% complete... Image deep copy: 26% complete... Image deep copy: 28% complete... Image deep copy: 29% complete... Image deep copy: 31% complete... Image deep copy: 32% complete... Image deep copy: 34% complete... Image deep copy: 35% complete... Image deep copy: 37% complete... Image deep copy: 39% complete... Image deep copy: 40% complete... Image deep copy: 42% complete... Image deep copy: 43% complete... Image deep copy: 45% complete... Image deep copy: 46% complete... Image deep copy: 48% complete... Image deep copy: 50% complete... Image deep copy: 51% complete... Image deep copy: 53% complete... Image deep copy: 54% complete... Image deep copy: 56% complete... Image deep copy: 57% complete... Image deep copy: 59% complete... Image deep copy: 60% complete... Image deep copy: 62% complete... Image deep copy: 64% complete... Image deep copy: 65% complete... Image deep copy: 67% complete... Image deep copy: 68% complete... Image deep copy: 70% complete... Image deep copy: 71% complete... Image deep copy: 73% complete... Image deep copy: 75% complete... Image deep copy: 76% complete... Image deep copy: 78% complete... Image deep copy: 79% complete... Image deep copy: 81% complete... Image deep copy: 82% complete... Image deep copy: 84% complete... Image deep copy: 85% complete... Image deep copy: 87% complete... Image deep copy: 89% complete... Image deep copy: 90% complete... Image deep copy: 92% complete... Image deep copy: 93% complete... Image deep copy: 95% complete... Image deep copy: 96% complete... Image deep copy: 98% complete... Image deep copy: 100% complete... Image deep copy: 100% complete...done. 2023-10-18T16:01:50.093 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg4 2023-10-18T16:01:50.093 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 128 MiB' 2023-10-18T16:01:50.133 INFO:tasks.workunit.client.0.smithi082.stdout: size 128 MiB in 32 objects 2023-10-18T16:01:50.133 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg5 2023-10-18T16:01:50.134 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 256 MiB' 2023-10-18T16:01:50.165 INFO:tasks.workunit.client.0.smithi082.stdout: size 256 MiB in 64 objects 2023-10-18T16:01:50.166 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap ls testimg4 2023-10-18T16:01:50.166 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -v SNAPID 2023-10-18T16:01:50.167 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:01:50.167 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 1 2023-10-18T16:01:50.198 INFO:tasks.workunit.client.0.smithi082.stdout:1 2023-10-18T16:01:50.199 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap ls testimg4 2023-10-18T16:01:50.199 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '.*snap1.*' 2023-10-18T16:01:50.231 INFO:tasks.workunit.client.0.smithi082.stdout: 18 snap1 256 MiB Wed Oct 18 16:01:49 2023 2023-10-18T16:01:50.231 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd export testimg1 /tmp/img1.new 2023-10-18T16:01:50.306 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-10-18T16:01:50.309 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd export testimg2 /tmp/img2.new 2023-10-18T16:01:50.408 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-10-18T16:01:50.411 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd export testimg3 /tmp/img3.new 2023-10-18T16:01:50.480 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-10-18T16:01:50.483 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd export testimg-diff1 /tmp/img-diff1.new 2023-10-18T16:01:50.552 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-10-18T16:01:50.555 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd export testimg-diff2 /tmp/img-diff2.new 2023-10-18T16:01:50.672 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-10-18T16:01:50.676 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd export testimg-diff3 /tmp/img-diff3.new 2023-10-18T16:01:50.742 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-10-18T16:01:50.746 INFO:tasks.workunit.client.0.smithi082.stderr:+ cmp /tmp/img2 /tmp/img2.new 2023-10-18T16:01:50.942 INFO:tasks.workunit.client.0.smithi082.stderr:+ cmp /tmp/img3 /tmp/img3.new 2023-10-18T16:01:51.037 INFO:tasks.workunit.client.0.smithi082.stderr:+ cmp /tmp/img2 /tmp/img-diff2.new 2023-10-18T16:01:51.194 INFO:tasks.workunit.client.0.smithi082.stderr:+ cmp /tmp/img3 /tmp/img-diff3.new 2023-10-18T16:01:51.271 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap rollback --snap=snap1 testimg1 2023-10-18T16:01:51.356 INFO:tasks.workunit.client.0.smithi082.stderr: Rolling back to snapshot: 1% complete... Rolling back to snapshot: 3% complete... Rolling back to snapshot: 4% complete... Rolling back to snapshot: 6% complete... Rolling back to snapshot: 7% complete... Rolling back to snapshot: 9% complete... Rolling back to snapshot: 10% complete... Rolling back to snapshot: 12% complete... Rolling back to snapshot: 14% complete... Rolling back to snapshot: 15% complete... Rolling back to snapshot: 17% complete... Rolling back to snapshot: 18% complete... Rolling back to snapshot: 20% complete... Rolling back to snapshot: 21% complete... Rolling back to snapshot: 23% complete... Rolling back to snapshot: 25% complete... Rolling back to snapshot: 26% complete... Rolling back to snapshot: 28% complete... Rolling back to snapshot: 29% complete... Rolling back to snapshot: 31% complete... Rolling back to snapshot: 32% complete... Rolling back to snapshot: 34% complete... Rolling back to snapshot: 35% complete... Rolling back to snapshot: 37% complete... Rolling back to snapshot: 39% complete... Rolling back to snapshot: 40% complete... Rolling back to snapshot: 42% complete... Rolling back to snapshot: 43% complete... Rolling back to snapshot: 45% complete... Rolling back to snapshot: 46% complete... Rolling back to snapshot: 48% complete... Rolling back to snapshot: 50% complete... Rolling back to snapshot: 51% complete... Rolling back to snapshot: 53% complete... Rolling back to snapshot: 54% complete... Rolling back to snapshot: 56% complete... Rolling back to snapshot: 57% complete... Rolling back to snapshot: 59% complete... Rolling back to snapshot: 60% complete... Rolling back to snapshot: 62% complete... Rolling back to snapshot: 64% complete... Rolling back to snapshot: 65% complete... Rolling back to snapshot: 67% complete... Rolling back to snapshot: 68% complete... Rolling back to snapshot: 70% complete... Rolling back to snapshot: 71% complete... Rolling back to snapshot: 73% complete... Rolling back to snapshot: 75% complete... Rolling back to snapshot: 76% complete... Rolling back to snapshot: 78% complete... Rolling back to snapshot: 79% complete... Rolling back to snapshot: 81% complete... Rolling back to snapshot: 82% complete... Rolling back to snapshot: 84% complete... Rolling back to snapshot: 85% complete... Rolling back to snapshot: 87% complete... Rolling back to snapshot: 89% complete... Rolling back to snapshot: 90% complete... Rolling back to snapshot: 92% complete... Rolling back to snapshot: 93% complete... Rolling back to snapshot: 95% complete... Rolling back to snapshot: 96% complete... Rolling back to snapshot: 98% complete... Rolling back to snapshot: 100% complete...done. 2023-10-18T16:01:51.363 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap rollback --snap=snap1 testimg-diff1 2023-10-18T16:01:51.458 INFO:tasks.workunit.client.0.smithi082.stderr: Rolling back to snapshot: 1% complete... Rolling back to snapshot: 3% complete... Rolling back to snapshot: 4% complete... Rolling back to snapshot: 6% complete... Rolling back to snapshot: 7% complete... Rolling back to snapshot: 9% complete... Rolling back to snapshot: 10% complete... Rolling back to snapshot: 12% complete... Rolling back to snapshot: 14% complete... Rolling back to snapshot: 15% complete... Rolling back to snapshot: 17% complete... Rolling back to snapshot: 18% complete... Rolling back to snapshot: 20% complete... Rolling back to snapshot: 21% complete... Rolling back to snapshot: 23% complete... Rolling back to snapshot: 25% complete... Rolling back to snapshot: 26% complete... Rolling back to snapshot: 28% complete... Rolling back to snapshot: 29% complete... Rolling back to snapshot: 31% complete... Rolling back to snapshot: 32% complete... Rolling back to snapshot: 34% complete... Rolling back to snapshot: 35% complete... Rolling back to snapshot: 37% complete... Rolling back to snapshot: 39% complete... Rolling back to snapshot: 40% complete... Rolling back to snapshot: 42% complete... Rolling back to snapshot: 43% complete... Rolling back to snapshot: 45% complete... Rolling back to snapshot: 46% complete... Rolling back to snapshot: 48% complete... Rolling back to snapshot: 50% complete... Rolling back to snapshot: 51% complete... Rolling back to snapshot: 53% complete... Rolling back to snapshot: 54% complete... Rolling back to snapshot: 56% complete... Rolling back to snapshot: 57% complete... Rolling back to snapshot: 59% complete... Rolling back to snapshot: 60% complete... Rolling back to snapshot: 62% complete... Rolling back to snapshot: 64% complete... Rolling back to snapshot: 65% complete... Rolling back to snapshot: 67% complete... Rolling back to snapshot: 68% complete... Rolling back to snapshot: 70% complete... Rolling back to snapshot: 71% complete... Rolling back to snapshot: 73% complete... Rolling back to snapshot: 75% complete... Rolling back to snapshot: 76% complete... Rolling back to snapshot: 78% complete... Rolling back to snapshot: 79% complete... Rolling back to snapshot: 81% complete... Rolling back to snapshot: 82% complete... Rolling back to snapshot: 84% complete... Rolling back to snapshot: 85% complete... Rolling back to snapshot: 87% complete... Rolling back to snapshot: 89% complete... Rolling back to snapshot: 90% complete... Rolling back to snapshot: 92% complete... Rolling back to snapshot: 93% complete... Rolling back to snapshot: 95% complete... Rolling back to snapshot: 96% complete... Rolling back to snapshot: 98% complete... Rolling back to snapshot: 100% complete...done. 2023-10-18T16:01:51.467 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg1 2023-10-18T16:01:51.467 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 256 MiB' 2023-10-18T16:01:51.497 INFO:tasks.workunit.client.0.smithi082.stdout: size 256 MiB in 64 objects 2023-10-18T16:01:51.498 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg-diff1 2023-10-18T16:01:51.499 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 256 MiB' 2023-10-18T16:01:51.533 INFO:tasks.workunit.client.0.smithi082.stdout: size 256 MiB in 64 objects 2023-10-18T16:01:51.533 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd export testimg1 /tmp/img1.snap1 2023-10-18T16:01:51.628 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-10-18T16:01:51.631 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd export testimg-diff1 /tmp/img-diff1.snap1 2023-10-18T16:01:51.734 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-10-18T16:01:51.737 INFO:tasks.workunit.client.0.smithi082.stderr:+ cmp /tmp/img2 /tmp/img1.snap1 2023-10-18T16:01:51.886 INFO:tasks.workunit.client.0.smithi082.stderr:+ cmp /tmp/img2 /tmp/img-diff1.snap1 2023-10-18T16:01:52.034 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm testimg2 2023-10-18T16:01:52.146 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 48% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 87% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 98% complete...2023-10-18T16:01:52.142+0000 7f9e3f622700 0 -- 172.21.15.82:0/3775719982 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x55ea29ec1a10 msgr2=0x55ea29e74e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:01:52.165 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:01:52.167 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm testimg3 2023-10-18T16:01:52.270 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2023-10-18T16:01:52.272 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create testimg2 -s 0 2023-10-18T16:01:52.299 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:52.295+0000 7f4390c30380 -1 librbd: Forced V1 image creation. 2023-10-18T16:01:52.305 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd cp testimg2 testimg3 2023-10-18T16:01:52.354 INFO:tasks.workunit.client.0.smithi082.stderr: Image copy: 100% complete...done. 2023-10-18T16:01:52.356 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd deep cp testimg2 testimg6 2023-10-18T16:01:52.420 INFO:tasks.workunit.client.0.smithi082.stderr: Image deep copy: 100% complete...done. 2023-10-18T16:01:52.422 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap rm --snap=snap1 testimg1 2023-10-18T16:01:53.054 INFO:tasks.workunit.client.0.smithi082.stderr: Removing snap: 100% complete...done. 2023-10-18T16:01:53.059 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap rm --snap=snap1 testimg-diff1 2023-10-18T16:01:54.046 INFO:tasks.workunit.client.0.smithi082.stderr: Removing snap: 100% complete...done. 2023-10-18T16:01:54.054 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info --snap=snap1 testimg1 2023-10-18T16:01:54.054 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'error setting snapshot context: (2) No such file or directory' 2023-10-18T16:01:54.093 INFO:tasks.workunit.client.0.smithi082.stdout:error setting snapshot context: (2) No such file or directory 2023-10-18T16:01:54.093 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info --snap=snap1 testimg-diff1 2023-10-18T16:01:54.094 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'error setting snapshot context: (2) No such file or directory' 2023-10-18T16:01:54.124 INFO:tasks.workunit.client.0.smithi082.stdout:error setting snapshot context: (2) No such file or directory 2023-10-18T16:01:54.125 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd sparsify testimg1 2023-10-18T16:01:54.222 INFO:tasks.workunit.client.0.smithi082.stderr: Image sparsify: 1% complete... Image sparsify: 3% complete... Image sparsify: 4% complete... Image sparsify: 6% complete... Image sparsify: 7% complete... Image sparsify: 9% complete... Image sparsify: 10% complete... Image sparsify: 12% complete... Image sparsify: 14% complete... Image sparsify: 15% complete... Image sparsify: 17% complete... Image sparsify: 18% complete... Image sparsify: 20% complete... Image sparsify: 21% complete... Image sparsify: 23% complete... Image sparsify: 25% complete... Image sparsify: 26% complete... Image sparsify: 28% complete... Image sparsify: 29% complete... Image sparsify: 31% complete... Image sparsify: 32% complete... Image sparsify: 34% complete... Image sparsify: 35% complete... Image sparsify: 37% complete... Image sparsify: 39% complete... Image sparsify: 40% complete... Image sparsify: 42% complete... Image sparsify: 43% complete... Image sparsify: 45% complete... Image sparsify: 46% complete... Image sparsify: 48% complete... Image sparsify: 50% complete... Image sparsify: 51% complete... Image sparsify: 53% complete... Image sparsify: 54% complete... Image sparsify: 56% complete... Image sparsify: 57% complete... Image sparsify: 59% complete... Image sparsify: 60% complete... Image sparsify: 62% complete... Image sparsify: 64% complete... Image sparsify: 65% complete... Image sparsify: 67% complete... Image sparsify: 68% complete... Image sparsify: 70% complete... Image sparsify: 71% complete... Image sparsify: 73% complete... Image sparsify: 75% complete... Image sparsify: 76% complete... Image sparsify: 78% complete... Image sparsify: 79% complete... Image sparsify: 81% complete... Image sparsify: 82% complete... Image sparsify: 84% complete... Image sparsify: 85% complete... Image sparsify: 87% complete... Image sparsify: 89% complete... Image sparsify: 90% complete... Image sparsify: 92% complete... Image sparsify: 93% complete... Image sparsify: 95% complete... Image sparsify: 96% complete... Image sparsify: 98% complete... Image sparsify: 100% complete...done. 2023-10-18T16:01:54.226 INFO:tasks.workunit.client.0.smithi082.stderr:+ remove_images 2023-10-18T16:01:54.227 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:54.346 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:54.422 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:54.542 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:55.144 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:56.202 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:56.320 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:56.435 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:56.603 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:56.745 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:56.820 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:56.896 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:56.974 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:57.047 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:57.120 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:57.198 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:57.273 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:57.348 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:57.427 INFO:tasks.workunit.client.0.smithi082.stderr:+ rm -f /tmp/img1 /tmp/img1.new /tmp/img2 /tmp/img2.new /tmp/img3 /tmp/img3.new /tmp/img-diff1.new /tmp/img-diff2.new /tmp/img-diff3.new /tmp/img1.snap1 /tmp/img1.snap1 /tmp/img-diff1.snap1 2023-10-18T16:01:57.543 INFO:tasks.workunit.client.0.smithi082.stderr:+ test_locking 2023-10-18T16:01:57.543 INFO:tasks.workunit.client.0.smithi082.stdout:testing locking... 2023-10-18T16:01:57.544 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'testing locking...' 2023-10-18T16:01:57.544 INFO:tasks.workunit.client.0.smithi082.stderr:+ remove_images 2023-10-18T16:01:57.544 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:57.617 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:57.691 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:57.769 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:57.845 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:57.931 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:58.006 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:58.080 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:58.156 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:58.235 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:58.309 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:58.385 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:58.462 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:58.538 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:58.613 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:58.689 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:58.762 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:58.835 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:01:58.909 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create -s 1 test1 2023-10-18T16:01:58.935 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:01:58.931+0000 7f735e9c6380 -1 librbd: Forced V1 image creation. 2023-10-18T16:01:58.943 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd lock list test1 2023-10-18T16:01:58.943 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:01:58.943 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^0$' 2023-10-18T16:01:58.973 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:01:58.974 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd lock add test1 id 2023-10-18T16:01:59.011 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd lock list test1 2023-10-18T16:01:59.011 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep ' 1 ' 2023-10-18T16:01:59.040 INFO:tasks.workunit.client.0.smithi082.stdout:There is 1 exclusive lock on this image. 2023-10-18T16:01:59.041 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd lock list test1 2023-10-18T16:01:59.042 INFO:tasks.workunit.client.0.smithi082.stderr:++ tail -n 1 2023-10-18T16:01:59.042 INFO:tasks.workunit.client.0.smithi082.stderr:++ awk '{print $1;}' 2023-10-18T16:01:59.072 INFO:tasks.workunit.client.0.smithi082.stderr:+ LOCKER=client.7773 2023-10-18T16:01:59.073 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd lock remove test1 id client.7773 2023-10-18T16:02:00.070 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd lock list test1 2023-10-18T16:02:00.070 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:00.070 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^0$' 2023-10-18T16:02:00.101 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:02:00.102 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd lock add test1 id --shared tag 2023-10-18T16:02:00.142 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd lock list test1 2023-10-18T16:02:00.142 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep ' 1 ' 2023-10-18T16:02:00.171 INFO:tasks.workunit.client.0.smithi082.stdout:There is 1 shared lock on this image. 2023-10-18T16:02:00.172 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd lock add test1 id --shared tag 2023-10-18T16:02:00.211 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd lock list test1 2023-10-18T16:02:00.211 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep ' 2 ' 2023-10-18T16:02:00.241 INFO:tasks.workunit.client.0.smithi082.stdout:There are 2 shared locks on this image. 2023-10-18T16:02:00.241 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd lock add test1 id2 --shared tag 2023-10-18T16:02:00.279 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd lock list test1 2023-10-18T16:02:00.280 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep ' 3 ' 2023-10-18T16:02:00.310 INFO:tasks.workunit.client.0.smithi082.stdout:There are 3 shared locks on this image. 2023-10-18T16:02:00.310 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd lock list test1 2023-10-18T16:02:00.311 INFO:tasks.workunit.client.0.smithi082.stderr:+ tail -n 1 2023-10-18T16:02:00.311 INFO:tasks.workunit.client.0.smithi082.stderr:+ awk '{print $2, $1;}' 2023-10-18T16:02:00.311 INFO:tasks.workunit.client.0.smithi082.stderr:+ xargs rbd lock remove test1 2023-10-18T16:02:01.075 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info test1 2023-10-18T16:02:01.076 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -qE 'features:.*exclusive' 2023-10-18T16:02:01.107 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test1 2023-10-18T16:02:01.145 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:02:01.148 INFO:tasks.workunit.client.0.smithi082.stderr:+ test_thick_provision 2023-10-18T16:02:01.148 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'testing thick provision...' 2023-10-18T16:02:01.148 INFO:tasks.workunit.client.0.smithi082.stdout:testing thick provision... 2023-10-18T16:02:01.149 INFO:tasks.workunit.client.0.smithi082.stderr:+ remove_images 2023-10-18T16:02:01.149 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:01.224 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:01.502 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:01.579 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:01.654 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:01.729 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:01.802 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:01.875 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:01.952 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:02.027 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:02.104 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:02.182 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:02.257 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:02.334 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:02.406 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:02.480 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:02.554 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:02.628 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:02.703 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --thick-provision -s 64M test1 2023-10-18T16:02:02.729 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:02:02.726+0000 7fdc10617380 -1 librbd: Forced V1 image creation. 2023-10-18T16:02:02.977 INFO:tasks.workunit.client.0.smithi082.stderr: Thick provisioning: 6% complete... Thick provisioning: 12% complete... Thick provisioning: 18% complete... Thick provisioning: 25% complete... Thick provisioning: 31% complete... Thick provisioning: 37% complete... Thick provisioning: 43% complete... Thick provisioning: 50% complete... Thick provisioning: 56% complete... Thick provisioning: 62% complete... Thick provisioning: 68% complete... Thick provisioning: 75% complete... Thick provisioning: 81% complete... Thick provisioning: 87% complete... Thick provisioning: 93% complete... Thick provisioning: 100% complete... Thick provisioning: 100% complete...done. 2023-10-18T16:02:02.982 INFO:tasks.workunit.client.0.smithi082.stderr:+ count=0 2023-10-18T16:02:02.982 INFO:tasks.workunit.client.0.smithi082.stderr:+ ret= 2023-10-18T16:02:02.982 INFO:tasks.workunit.client.0.smithi082.stderr:+ '[' 0 -lt 10 ']' 2023-10-18T16:02:02.982 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd du 2023-10-18T16:02:02.983 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test1 2023-10-18T16:02:02.983 INFO:tasks.workunit.client.0.smithi082.stderr:+ tr -s ' ' 2023-10-18T16:02:02.983 INFO:tasks.workunit.client.0.smithi082.stderr:+ cut -d ' ' -f 4-5 2023-10-18T16:02:02.984 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^64 MiB' 2023-10-18T16:02:03.009 INFO:tasks.workunit.client.0.smithi082.stderr:warning: fast-diff map is not enabled for test1. operation may be slow. 2023-10-18T16:02:03.013 INFO:tasks.workunit.client.0.smithi082.stdout:64 MiB 2023-10-18T16:02:03.014 INFO:tasks.workunit.client.0.smithi082.stderr:+ ret=0 2023-10-18T16:02:03.014 INFO:tasks.workunit.client.0.smithi082.stderr:+ '[' 0 = 0 ']' 2023-10-18T16:02:03.015 INFO:tasks.workunit.client.0.smithi082.stderr:+ break 2023-10-18T16:02:03.015 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd du 2023-10-18T16:02:03.041 INFO:tasks.workunit.client.0.smithi082.stderr:warning: fast-diff map is not enabled for test1. operation may be slow. 2023-10-18T16:02:03.043 INFO:tasks.workunit.client.0.smithi082.stdout:NAME PROVISIONED USED 2023-10-18T16:02:03.043 INFO:tasks.workunit.client.0.smithi082.stdout:test1 64 MiB 64 MiB 2023-10-18T16:02:03.045 INFO:tasks.workunit.client.0.smithi082.stderr:+ '[' 0 '!=' 0 ']' 2023-10-18T16:02:03.046 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test1 2023-10-18T16:02:03.116 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 6% complete... Removing image: 12% complete... Removing image: 18% complete... Removing image: 25% complete... Removing image: 31% complete... Removing image: 37% complete... Removing image: 43% complete... Removing image: 50% complete... Removing image: 56% complete... Removing image: 62% complete... Removing image: 68% complete... Removing image: 75% complete... Removing image: 81% complete... Removing image: 87% complete... Removing image: 93% complete... Removing image: 100% complete...done. 2023-10-18T16:02:03.120 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:02:03.120 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test1 2023-10-18T16:02:03.120 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:03.121 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^0$' 2023-10-18T16:02:03.147 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:02:03.147 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --thick-provision -s 4G test1 2023-10-18T16:02:03.174 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:02:03.170+0000 7fc298f41380 -1 librbd: Forced V1 image creation. 2023-10-18T16:02:04.115 INFO:tasks.workunit.client.0.smithi082.stderr: Thick provisioning: 1% complete... Thick provisioning: 2% complete... Thick provisioning: 3% complete... Thick provisioning: 4% complete... Thick provisioning: 5% complete... Thick provisioning: 6% complete... Thick provisioning: 7% complete... Thick provisioning: 8% complete... Thick provisioning: 9% complete... Thick provisioning: 10% complete... Thick provisioning: 11% complete...2023-10-18T16:02:04.112+0000 7fc288d75700 0 -- 172.21.15.82:0/3764446389 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x55d8b336bf50 msgr2=0x55d8b340ad90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:02:04.317 INFO:tasks.workunit.client.0.smithi082.stderr: Thick provisioning: 12% complete... Thick provisioning: 13% complete...2023-10-18T16:02:04.314+0000 7fc288d75700 0 -- 172.21.15.82:0/3764446389 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7fc268036500 msgr2=0x7fc268038930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:02:12.884 INFO:tasks.workunit.client.0.smithi082.stderr: Thick provisioning: 14% complete... Thick provisioning: 15% complete... Thick provisioning: 16% complete... Thick provisioning: 17% complete... Thick provisioning: 18% complete... Thick provisioning: 19% complete... Thick provisioning: 20% complete... Thick provisioning: 21% complete... Thick provisioning: 22% complete... Thick provisioning: 23% complete... Thick provisioning: 24% complete... Thick provisioning: 25% complete... Thick provisioning: 26% complete... Thick provisioning: 27% complete... Thick provisioning: 28% complete... Thick provisioning: 29% complete... Thick provisioning: 30% complete... Thick provisioning: 31% complete... Thick provisioning: 32% complete... Thick provisioning: 33% complete... Thick provisioning: 34% complete... Thick provisioning: 35% complete... Thick provisioning: 36% complete... Thick provisioning: 37% complete... Thick provisioning: 38% complete... Thick provisioning: 39% complete... Thick provisioning: 40% complete... Thick provisioning: 41% complete... Thick provisioning: 42% complete... Thick provisioning: 43% complete... Thick provisioning: 44% complete... Thick provisioning: 45% complete... Thick provisioning: 46% complete... Thick provisioning: 47% complete... Thick provisioning: 48% complete... Thick provisioning: 49% complete... Thick provisioning: 50% complete... Thick provisioning: 51% complete... Thick provisioning: 52% complete... Thick provisioning: 53% complete... Thick provisioning: 54% complete... Thick provisioning: 55% complete... Thick provisioning: 56% complete... Thick provisioning: 57% complete... Thick provisioning: 58% complete... Thick provisioning: 59% complete... Thick provisioning: 60% complete... Thick provisioning: 61% complete... Thick provisioning: 62% complete... Thick provisioning: 63% complete... Thick provisioning: 64% complete... Thick provisioning: 65% complete... Thick provisioning: 66% complete... Thick provisioning: 67% complete... Thick provisioning: 68% complete... Thick provisioning: 69% complete... Thick provisioning: 70% complete... Thick provisioning: 71% complete... Thick provisioning: 72% complete... Thick provisioning: 73% complete... Thick provisioning: 74% complete... Thick provisioning: 75% complete... Thick provisioning: 76% complete... Thick provisioning: 77% complete... Thick provisioning: 78% complete... Thick provisioning: 79% complete... Thick provisioning: 80% complete... Thick provisioning: 81% complete... Thick provisioning: 82% complete... Thick provisioning: 83% complete... Thick provisioning: 84% complete... Thick provisioning: 85% complete... Thick provisioning: 86% complete... Thick provisioning: 87% complete... Thick provisioning: 88% complete... Thick provisioning: 89% complete... Thick provisioning: 90% complete... Thick provisioning: 91% complete... Thick provisioning: 92% complete... Thick provisioning: 93% complete... Thick provisioning: 94% complete... Thick provisioning: 95% complete... Thick provisioning: 96% complete... Thick provisioning: 97% complete... Thick provisioning: 98% complete... Thick provisioning: 99% complete... Thick provisioning: 100% complete... Thick provisioning: 100% complete...done. 2023-10-18T16:02:12.889 INFO:tasks.workunit.client.0.smithi082.stderr:+ count=0 2023-10-18T16:02:12.889 INFO:tasks.workunit.client.0.smithi082.stderr:+ ret= 2023-10-18T16:02:12.889 INFO:tasks.workunit.client.0.smithi082.stderr:+ '[' 0 -lt 10 ']' 2023-10-18T16:02:12.889 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd du 2023-10-18T16:02:12.890 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test1 2023-10-18T16:02:12.890 INFO:tasks.workunit.client.0.smithi082.stderr:+ tr -s ' ' 2023-10-18T16:02:12.890 INFO:tasks.workunit.client.0.smithi082.stderr:+ cut -d ' ' -f 4-5 2023-10-18T16:02:12.890 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^4 GiB' 2023-10-18T16:02:12.916 INFO:tasks.workunit.client.0.smithi082.stderr:warning: fast-diff map is not enabled for test1. operation may be slow. 2023-10-18T16:02:12.929 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:02:12.925+0000 7f1a9082e700 0 -- 172.21.15.82:0/4225232186 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x55856ba74710 msgr2=0x55856ba84580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:02:12.931 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:02:12.927+0000 7f1a9082e700 0 -- 172.21.15.82:0/4225232186 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x7f1a740364f0 msgr2=0x7f1a74038920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:02:13.096 INFO:tasks.workunit.client.0.smithi082.stdout:4 GiB 2023-10-18T16:02:13.097 INFO:tasks.workunit.client.0.smithi082.stderr:+ ret=0 2023-10-18T16:02:13.098 INFO:tasks.workunit.client.0.smithi082.stderr:+ '[' 0 = 0 ']' 2023-10-18T16:02:13.098 INFO:tasks.workunit.client.0.smithi082.stderr:+ break 2023-10-18T16:02:13.098 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd du 2023-10-18T16:02:13.123 INFO:tasks.workunit.client.0.smithi082.stderr:warning: fast-diff map is not enabled for test1. operation may be slow. 2023-10-18T16:02:13.135 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:02:13.132+0000 7f42dabc2700 0 -- 172.21.15.82:0/3932659332 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x562741b81730 msgr2=0x562741b74b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:02:13.138 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:02:13.134+0000 7f42da3c1700 0 -- 172.21.15.82:0/3932659332 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x562741b98cb0 msgr2=0x562741b9b0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:02:13.261 INFO:tasks.workunit.client.0.smithi082.stdout:NAME PROVISIONED USED 2023-10-18T16:02:13.261 INFO:tasks.workunit.client.0.smithi082.stdout:test1 4 GiB 4 GiB 2023-10-18T16:02:13.263 INFO:tasks.workunit.client.0.smithi082.stderr:+ '[' 0 '!=' 0 ']' 2023-10-18T16:02:13.264 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test1 2023-10-18T16:02:13.391 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete...2023-10-18T16:02:13.387+0000 7f2cfc41b700 0 -- 172.21.15.82:0/2551537288 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x555cb89d5eb0 msgr2=0x555cb89d82a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:02:13.408 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 11% complete... Removing image: 12% complete...2023-10-18T16:02:13.404+0000 7f2cfc41b700 0 -- 172.21.15.82:0/2551537288 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7f2ce00365d0 msgr2=0x7f2ce0038a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:02:14.154 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2023-10-18T16:02:14.157 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:02:14.157 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test1 2023-10-18T16:02:14.157 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:14.158 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^0$' 2023-10-18T16:02:14.182 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:02:14.183 INFO:tasks.workunit.client.0.smithi082.stderr:+ RBD_CREATE_ARGS='--image-format 2' 2023-10-18T16:02:14.183 INFO:tasks.workunit.client.0.smithi082.stderr:+ test_others 2023-10-18T16:02:14.184 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'testing import, export, resize, and snapshots...' 2023-10-18T16:02:14.184 INFO:tasks.workunit.client.0.smithi082.stdout:testing import, export, resize, and snapshots... 2023-10-18T16:02:14.185 INFO:tasks.workunit.client.0.smithi082.stderr:+ TMP_FILES='/tmp/img1 /tmp/img1.new /tmp/img2 /tmp/img2.new /tmp/img3 /tmp/img3.new /tmp/img-diff1.new /tmp/img-diff2.new /tmp/img-diff3.new /tmp/img1.snap1 /tmp/img1.snap1 /tmp/img-diff1.snap1' 2023-10-18T16:02:14.185 INFO:tasks.workunit.client.0.smithi082.stderr:+ remove_images 2023-10-18T16:02:14.185 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:14.257 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:14.330 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:14.404 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:14.477 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:14.552 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:14.625 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:14.699 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:14.774 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:14.848 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:14.923 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:14.997 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:15.071 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:15.145 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:15.219 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:15.294 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:15.369 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:15.443 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:15.518 INFO:tasks.workunit.client.0.smithi082.stderr:+ rm -f /tmp/img1 /tmp/img1.new /tmp/img2 /tmp/img2.new /tmp/img3 /tmp/img3.new /tmp/img-diff1.new /tmp/img-diff2.new /tmp/img-diff3.new /tmp/img1.snap1 /tmp/img1.snap1 /tmp/img-diff1.snap1 2023-10-18T16:02:15.518 INFO:tasks.workunit.client.0.smithi082.stderr:+ dd if=/bin/sh of=/tmp/img1 bs=1k count=1 seek=10 2023-10-18T16:02:15.519 INFO:tasks.workunit.client.0.smithi082.stderr:1+0 records in 2023-10-18T16:02:15.519 INFO:tasks.workunit.client.0.smithi082.stderr:1+0 records out 2023-10-18T16:02:15.519 INFO:tasks.workunit.client.0.smithi082.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 4.2752e-05 s, 24.0 MB/s 2023-10-18T16:02:15.520 INFO:tasks.workunit.client.0.smithi082.stderr:+ dd if=/bin/dd of=/tmp/img1 bs=1k count=10 seek=100 2023-10-18T16:02:15.521 INFO:tasks.workunit.client.0.smithi082.stderr:10+0 records in 2023-10-18T16:02:15.521 INFO:tasks.workunit.client.0.smithi082.stderr:10+0 records out 2023-10-18T16:02:15.521 INFO:tasks.workunit.client.0.smithi082.stderr:10240 bytes (10 kB, 10 KiB) copied, 5.1579e-05 s, 199 MB/s 2023-10-18T16:02:15.521 INFO:tasks.workunit.client.0.smithi082.stderr:+ dd if=/bin/rm of=/tmp/img1 bs=1k count=100 seek=1000 2023-10-18T16:02:15.521 INFO:tasks.workunit.client.0.smithi082.stderr:70+1 records in 2023-10-18T16:02:15.522 INFO:tasks.workunit.client.0.smithi082.stderr:70+1 records out 2023-10-18T16:02:15.522 INFO:tasks.workunit.client.0.smithi082.stderr:72216 bytes (72 kB, 71 KiB) copied, 0.000196471 s, 368 MB/s 2023-10-18T16:02:15.522 INFO:tasks.workunit.client.0.smithi082.stderr:+ dd if=/bin/ls of=/tmp/img1 bs=1k seek=10000 2023-10-18T16:02:15.523 INFO:tasks.workunit.client.0.smithi082.stderr:140+1 records in 2023-10-18T16:02:15.523 INFO:tasks.workunit.client.0.smithi082.stderr:140+1 records out 2023-10-18T16:02:15.523 INFO:tasks.workunit.client.0.smithi082.stderr:143368 bytes (143 kB, 140 KiB) copied, 0.000334383 s, 429 MB/s 2023-10-18T16:02:15.524 INFO:tasks.workunit.client.0.smithi082.stderr:+ dd if=/bin/ln of=/tmp/img1 bs=1k seek=100000 2023-10-18T16:02:15.524 INFO:tasks.workunit.client.0.smithi082.stderr:70+1 records in 2023-10-18T16:02:15.524 INFO:tasks.workunit.client.0.smithi082.stderr:70+1 records out 2023-10-18T16:02:15.524 INFO:tasks.workunit.client.0.smithi082.stderr:72320 bytes (72 kB, 71 KiB) copied, 0.000177442 s, 408 MB/s 2023-10-18T16:02:15.524 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd import --image-format 2 /tmp/img1 testimg1 2023-10-18T16:02:15.657 INFO:tasks.workunit.client.0.smithi082.stderr: Importing image: 4% complete... Importing image: 8% complete... Importing image: 12% complete... Importing image: 16% complete... Importing image: 20% complete... Importing image: 24% complete... Importing image: 28% complete... Importing image: 32% complete... Importing image: 36% complete... Importing image: 40% complete... Importing image: 45% complete... Importing image: 49% complete... Importing image: 53% complete... Importing image: 57% complete... Importing image: 61% complete... Importing image: 65% complete... Importing image: 69% complete... Importing image: 73% complete... Importing image: 77% complete... Importing image: 81% complete... Importing image: 85% complete... Importing image: 90% complete... Importing image: 94% complete... Importing image: 98% complete... Importing image: 100% complete...done. 2023-10-18T16:02:15.660 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd resize testimg1 --size=256 --allow-shrink 2023-10-18T16:02:15.704 INFO:tasks.workunit.client.0.smithi082.stderr: Resizing image: 100% complete...done. 2023-10-18T16:02:15.711 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd export testimg1 /tmp/img2 2023-10-18T16:02:15.814 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-10-18T16:02:15.817 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create testimg1 --snap=snap1 2023-10-18T16:02:15.976 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:02:15.984 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd resize testimg1 --size=128 2023-10-18T16:02:16.218 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: shrinking an image is only allowed with the --allow-shrink flag 2023-10-18T16:02:16.223 INFO:tasks.workunit.client.0.smithi082.stderr:+ true 2023-10-18T16:02:16.223 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd resize testimg1 --size=128 --allow-shrink 2023-10-18T16:02:16.264 INFO:tasks.workunit.client.0.smithi082.stderr: Resizing image: 50% complete... Resizing image: 51% complete... Resizing image: 53% complete... Resizing image: 54% complete... Resizing image: 56% complete... Resizing image: 57% complete... Resizing image: 59% complete... Resizing image: 60% complete... Resizing image: 62% complete... Resizing image: 64% complete... Resizing image: 65% complete... Resizing image: 67% complete... Resizing image: 68% complete... Resizing image: 70% complete... Resizing image: 71% complete... Resizing image: 73% complete... Resizing image: 75% complete... Resizing image: 76% complete... Resizing image: 78% complete... Resizing image: 79% complete... Resizing image: 81% complete... Resizing image: 82% complete... Resizing image: 84% complete... Resizing image: 85% complete... Resizing image: 87% complete... Resizing image: 89% complete... Resizing image: 90% complete... Resizing image: 92% complete... Resizing image: 93% complete... Resizing image: 95% complete... Resizing image: 96% complete... Resizing image: 98% complete... Resizing image: 100% complete...done. 2023-10-18T16:02:16.271 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd export testimg1 /tmp/img3 2023-10-18T16:02:16.342 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-10-18T16:02:16.345 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg1 2023-10-18T16:02:16.346 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 128 MiB' 2023-10-18T16:02:16.378 INFO:tasks.workunit.client.0.smithi082.stdout: size 128 MiB in 32 objects 2023-10-18T16:02:16.379 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info --snap=snap1 testimg1 2023-10-18T16:02:16.379 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 256 MiB' 2023-10-18T16:02:16.413 INFO:tasks.workunit.client.0.smithi082.stdout: size 256 MiB in 64 objects 2023-10-18T16:02:16.413 INFO:tasks.workunit.client.0.smithi082.stderr:+ rm -rf /tmp/diff-testimg1-1 /tmp/diff-testimg1-2 2023-10-18T16:02:16.414 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd export-diff testimg1 --snap=snap1 /tmp/diff-testimg1-1 2023-10-18T16:02:16.459 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 3% complete... Exporting image: 37% complete... Exporting image: 100% complete...done. 2023-10-18T16:02:16.462 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd export-diff testimg1 --from-snap=snap1 /tmp/diff-testimg1-2 2023-10-18T16:02:16.498 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 100% complete...done. 2023-10-18T16:02:16.501 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 --size=1 testimg-diff1 2023-10-18T16:02:16.541 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd import-diff --sparse-size 8K /tmp/diff-testimg1-1 testimg-diff1 2023-10-18T16:02:16.975 INFO:tasks.workunit.client.0.smithi082.stderr: Importing image diff: 22% complete... Importing image diff: 63% complete... Importing image diff: 99% complete... Importing image diff: 100% complete...done. 2023-10-18T16:02:16.982 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd import-diff --sparse-size 8K /tmp/diff-testimg1-2 testimg-diff1 2023-10-18T16:02:17.024 INFO:tasks.workunit.client.0.smithi082.stderr: Importing image diff: 68% complete... Importing image diff: 96% complete... Importing image diff: 100% complete...done. 2023-10-18T16:02:17.031 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg1 2023-10-18T16:02:17.032 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 128 MiB' 2023-10-18T16:02:17.064 INFO:tasks.workunit.client.0.smithi082.stdout: size 128 MiB in 32 objects 2023-10-18T16:02:17.064 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info --snap=snap1 testimg1 2023-10-18T16:02:17.065 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 256 MiB' 2023-10-18T16:02:17.098 INFO:tasks.workunit.client.0.smithi082.stdout: size 256 MiB in 64 objects 2023-10-18T16:02:17.099 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg-diff1 2023-10-18T16:02:17.099 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 128 MiB' 2023-10-18T16:02:17.131 INFO:tasks.workunit.client.0.smithi082.stdout: size 128 MiB in 32 objects 2023-10-18T16:02:17.132 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info --snap=snap1 testimg-diff1 2023-10-18T16:02:17.132 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 256 MiB' 2023-10-18T16:02:17.166 INFO:tasks.workunit.client.0.smithi082.stdout: size 256 MiB in 64 objects 2023-10-18T16:02:17.167 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd copy testimg1 --snap=snap1 testimg2 2023-10-18T16:02:17.262 INFO:tasks.workunit.client.0.smithi082.stderr: Image copy: 3% complete... Image copy: 37% complete... Image copy: 100% complete... Image copy: 100% complete...done. 2023-10-18T16:02:17.265 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd copy testimg1 testimg3 2023-10-18T16:02:17.366 INFO:tasks.workunit.client.0.smithi082.stderr: Image copy: 3% complete... Image copy: 6% complete... Image copy: 9% complete... Image copy: 12% complete... Image copy: 15% complete... Image copy: 18% complete... Image copy: 21% complete... Image copy: 25% complete... Image copy: 28% complete... Image copy: 31% complete... Image copy: 34% complete... Image copy: 37% complete... Image copy: 40% complete... Image copy: 43% complete... Image copy: 46% complete... Image copy: 50% complete... Image copy: 53% complete... Image copy: 56% complete... Image copy: 59% complete... Image copy: 62% complete... Image copy: 65% complete... Image copy: 68% complete... Image copy: 71% complete... Image copy: 75% complete... Image copy: 78% complete... Image copy: 81% complete... Image copy: 84% complete... Image copy: 87% complete... Image copy: 90% complete... Image copy: 93% complete... Image copy: 96% complete... Image copy: 100% complete... Image copy: 100% complete...done. 2023-10-18T16:02:17.369 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd copy testimg-diff1 --sparse-size 768K --snap=snap1 testimg-diff2 2023-10-18T16:02:17.460 INFO:tasks.workunit.client.0.smithi082.stderr: Image copy: 3% complete... Image copy: 37% complete... Image copy: 100% complete... Image copy: 100% complete...done. 2023-10-18T16:02:17.463 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd copy testimg-diff1 --sparse-size 768K testimg-diff3 2023-10-18T16:02:17.558 INFO:tasks.workunit.client.0.smithi082.stderr: Image copy: 3% complete... Image copy: 6% complete... Image copy: 9% complete... Image copy: 12% complete... Image copy: 15% complete... Image copy: 18% complete... Image copy: 21% complete... Image copy: 25% complete... Image copy: 28% complete... Image copy: 31% complete... Image copy: 34% complete... Image copy: 37% complete... Image copy: 40% complete... Image copy: 43% complete... Image copy: 46% complete... Image copy: 50% complete... Image copy: 53% complete... Image copy: 56% complete... Image copy: 59% complete... Image copy: 62% complete... Image copy: 65% complete... Image copy: 68% complete... Image copy: 71% complete... Image copy: 75% complete... Image copy: 78% complete... Image copy: 81% complete... Image copy: 84% complete... Image copy: 87% complete... Image copy: 90% complete... Image copy: 93% complete... Image copy: 96% complete... Image copy: 100% complete... Image copy: 100% complete...done. 2023-10-18T16:02:17.561 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg2 2023-10-18T16:02:17.562 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 256 MiB' 2023-10-18T16:02:17.593 INFO:tasks.workunit.client.0.smithi082.stdout: size 256 MiB in 64 objects 2023-10-18T16:02:17.594 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg3 2023-10-18T16:02:17.594 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 128 MiB' 2023-10-18T16:02:17.625 INFO:tasks.workunit.client.0.smithi082.stdout: size 128 MiB in 32 objects 2023-10-18T16:02:17.626 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg-diff2 2023-10-18T16:02:17.626 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 256 MiB' 2023-10-18T16:02:17.658 INFO:tasks.workunit.client.0.smithi082.stdout: size 256 MiB in 64 objects 2023-10-18T16:02:17.659 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg-diff3 2023-10-18T16:02:17.659 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 128 MiB' 2023-10-18T16:02:17.691 INFO:tasks.workunit.client.0.smithi082.stdout: size 128 MiB in 32 objects 2023-10-18T16:02:17.691 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd deep copy testimg1 testimg4 2023-10-18T16:02:17.996 INFO:tasks.workunit.client.0.smithi082.stderr: Image deep copy: 1% complete... Image deep copy: 3% complete... Image deep copy: 4% complete... Image deep copy: 6% complete... Image deep copy: 7% complete... Image deep copy: 9% complete... Image deep copy: 10% complete... Image deep copy: 12% complete... Image deep copy: 14% complete... Image deep copy: 15% complete... Image deep copy: 17% complete... Image deep copy: 18% complete... Image deep copy: 20% complete... Image deep copy: 21% complete... Image deep copy: 23% complete... Image deep copy: 25% complete... Image deep copy: 26% complete... Image deep copy: 28% complete... Image deep copy: 29% complete... Image deep copy: 31% complete... Image deep copy: 32% complete... Image deep copy: 34% complete... Image deep copy: 35% complete... Image deep copy: 37% complete... Image deep copy: 39% complete... Image deep copy: 40% complete... Image deep copy: 42% complete... Image deep copy: 43% complete... Image deep copy: 45% complete... Image deep copy: 46% complete... Image deep copy: 48% complete... Image deep copy: 50% complete... Image deep copy: 51% complete... Image deep copy: 53% complete... Image deep copy: 54% complete... Image deep copy: 56% complete... Image deep copy: 57% complete... Image deep copy: 59% complete... Image deep copy: 60% complete... Image deep copy: 62% complete... Image deep copy: 64% complete... Image deep copy: 65% complete... Image deep copy: 67% complete... Image deep copy: 68% complete... Image deep copy: 70% complete... Image deep copy: 71% complete... Image deep copy: 73% complete... Image deep copy: 75% complete... Image deep copy: 76% complete... Image deep copy: 78% complete... Image deep copy: 79% complete... Image deep copy: 81% complete... Image deep copy: 82% complete... Image deep copy: 84% complete... Image deep copy: 85% complete... Image deep copy: 87% complete... Image deep copy: 89% complete... Image deep copy: 90% complete... Image deep copy: 92% complete... Image deep copy: 93% complete... Image deep copy: 95% complete... Image deep copy: 96% complete... Image deep copy: 98% complete... Image deep copy: 100% complete... Image deep copy: 100% complete...done. 2023-10-18T16:02:17.998 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd deep copy testimg1 --snap=snap1 testimg5 2023-10-18T16:02:19.014 INFO:tasks.workunit.client.0.smithi082.stderr: Image deep copy: 1% complete... Image deep copy: 3% complete... Image deep copy: 4% complete... Image deep copy: 6% complete... Image deep copy: 7% complete... Image deep copy: 9% complete... Image deep copy: 10% complete... Image deep copy: 12% complete... Image deep copy: 14% complete... Image deep copy: 15% complete... Image deep copy: 17% complete... Image deep copy: 18% complete... Image deep copy: 20% complete... Image deep copy: 21% complete... Image deep copy: 23% complete... Image deep copy: 25% complete... Image deep copy: 26% complete... Image deep copy: 28% complete... Image deep copy: 29% complete... Image deep copy: 31% complete... Image deep copy: 32% complete... Image deep copy: 34% complete... Image deep copy: 35% complete... Image deep copy: 37% complete... Image deep copy: 39% complete... Image deep copy: 40% complete... Image deep copy: 42% complete... Image deep copy: 43% complete... Image deep copy: 45% complete... Image deep copy: 46% complete... Image deep copy: 48% complete... Image deep copy: 50% complete... Image deep copy: 51% complete... Image deep copy: 53% complete... Image deep copy: 54% complete... Image deep copy: 56% complete... Image deep copy: 57% complete... Image deep copy: 59% complete... Image deep copy: 60% complete... Image deep copy: 62% complete... Image deep copy: 64% complete... Image deep copy: 65% complete... Image deep copy: 67% complete... Image deep copy: 68% complete... Image deep copy: 70% complete... Image deep copy: 71% complete... Image deep copy: 73% complete... Image deep copy: 75% complete... Image deep copy: 76% complete... Image deep copy: 78% complete... Image deep copy: 79% complete... Image deep copy: 81% complete... Image deep copy: 82% complete... Image deep copy: 84% complete... Image deep copy: 85% complete... Image deep copy: 87% complete... Image deep copy: 89% complete... Image deep copy: 90% complete... Image deep copy: 92% complete... Image deep copy: 93% complete... Image deep copy: 95% complete... Image deep copy: 96% complete... Image deep copy: 98% complete... Image deep copy: 100% complete... Image deep copy: 100% complete...done. 2023-10-18T16:02:19.018 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg4 2023-10-18T16:02:19.018 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 128 MiB' 2023-10-18T16:02:19.052 INFO:tasks.workunit.client.0.smithi082.stdout: size 128 MiB in 32 objects 2023-10-18T16:02:19.053 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg5 2023-10-18T16:02:19.053 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 256 MiB' 2023-10-18T16:02:19.087 INFO:tasks.workunit.client.0.smithi082.stdout: size 256 MiB in 64 objects 2023-10-18T16:02:19.088 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap ls testimg4 2023-10-18T16:02:19.088 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -v SNAPID 2023-10-18T16:02:19.089 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:19.089 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 1 2023-10-18T16:02:19.120 INFO:tasks.workunit.client.0.smithi082.stdout:1 2023-10-18T16:02:19.121 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap ls testimg4 2023-10-18T16:02:19.121 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '.*snap1.*' 2023-10-18T16:02:19.154 INFO:tasks.workunit.client.0.smithi082.stdout: 26 snap1 256 MiB Wed Oct 18 16:02:17 2023 2023-10-18T16:02:19.154 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd export testimg1 /tmp/img1.new 2023-10-18T16:02:19.229 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-10-18T16:02:19.232 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd export testimg2 /tmp/img2.new 2023-10-18T16:02:19.339 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-10-18T16:02:19.342 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd export testimg3 /tmp/img3.new 2023-10-18T16:02:19.416 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-10-18T16:02:19.419 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd export testimg-diff1 /tmp/img-diff1.new 2023-10-18T16:02:19.486 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-10-18T16:02:19.490 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd export testimg-diff2 /tmp/img-diff2.new 2023-10-18T16:02:19.589 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-10-18T16:02:19.592 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd export testimg-diff3 /tmp/img-diff3.new 2023-10-18T16:02:19.670 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-10-18T16:02:19.672 INFO:tasks.workunit.client.0.smithi082.stderr:+ cmp /tmp/img2 /tmp/img2.new 2023-10-18T16:02:19.867 INFO:tasks.workunit.client.0.smithi082.stderr:+ cmp /tmp/img3 /tmp/img3.new 2023-10-18T16:02:19.962 INFO:tasks.workunit.client.0.smithi082.stderr:+ cmp /tmp/img2 /tmp/img-diff2.new 2023-10-18T16:02:20.121 INFO:tasks.workunit.client.0.smithi082.stderr:+ cmp /tmp/img3 /tmp/img-diff3.new 2023-10-18T16:02:20.200 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap rollback --snap=snap1 testimg1 2023-10-18T16:02:20.293 INFO:tasks.workunit.client.0.smithi082.stderr: Rolling back to snapshot: 1% complete... Rolling back to snapshot: 3% complete... Rolling back to snapshot: 4% complete... Rolling back to snapshot: 6% complete... Rolling back to snapshot: 7% complete... Rolling back to snapshot: 9% complete... Rolling back to snapshot: 10% complete... Rolling back to snapshot: 12% complete... Rolling back to snapshot: 14% complete... Rolling back to snapshot: 15% complete... Rolling back to snapshot: 17% complete... Rolling back to snapshot: 18% complete... Rolling back to snapshot: 20% complete... Rolling back to snapshot: 21% complete... Rolling back to snapshot: 23% complete... Rolling back to snapshot: 25% complete... Rolling back to snapshot: 26% complete... Rolling back to snapshot: 28% complete... Rolling back to snapshot: 29% complete... Rolling back to snapshot: 31% complete... Rolling back to snapshot: 32% complete... Rolling back to snapshot: 34% complete... Rolling back to snapshot: 35% complete... Rolling back to snapshot: 37% complete... Rolling back to snapshot: 39% complete... Rolling back to snapshot: 40% complete... Rolling back to snapshot: 42% complete... Rolling back to snapshot: 43% complete... Rolling back to snapshot: 45% complete... Rolling back to snapshot: 46% complete... Rolling back to snapshot: 48% complete... Rolling back to snapshot: 50% complete... Rolling back to snapshot: 51% complete... Rolling back to snapshot: 53% complete... Rolling back to snapshot: 54% complete... Rolling back to snapshot: 56% complete... Rolling back to snapshot: 57% complete... Rolling back to snapshot: 59% complete... Rolling back to snapshot: 60% complete... Rolling back to snapshot: 62% complete... Rolling back to snapshot: 64% complete... Rolling back to snapshot: 65% complete... Rolling back to snapshot: 67% complete... Rolling back to snapshot: 68% complete... Rolling back to snapshot: 70% complete... Rolling back to snapshot: 71% complete... Rolling back to snapshot: 73% complete... Rolling back to snapshot: 75% complete... Rolling back to snapshot: 76% complete... Rolling back to snapshot: 78% complete... Rolling back to snapshot: 79% complete... Rolling back to snapshot: 81% complete... Rolling back to snapshot: 82% complete... Rolling back to snapshot: 84% complete... Rolling back to snapshot: 85% complete... Rolling back to snapshot: 87% complete... Rolling back to snapshot: 89% complete... Rolling back to snapshot: 90% complete... Rolling back to snapshot: 92% complete... Rolling back to snapshot: 93% complete... Rolling back to snapshot: 95% complete... Rolling back to snapshot: 96% complete... Rolling back to snapshot: 98% complete... Rolling back to snapshot: 100% complete...done. 2023-10-18T16:02:20.300 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap rollback --snap=snap1 testimg-diff1 2023-10-18T16:02:20.396 INFO:tasks.workunit.client.0.smithi082.stderr: Rolling back to snapshot: 1% complete... Rolling back to snapshot: 3% complete... Rolling back to snapshot: 4% complete... Rolling back to snapshot: 6% complete... Rolling back to snapshot: 7% complete... Rolling back to snapshot: 9% complete... Rolling back to snapshot: 10% complete... Rolling back to snapshot: 12% complete... Rolling back to snapshot: 14% complete... Rolling back to snapshot: 15% complete... Rolling back to snapshot: 17% complete... Rolling back to snapshot: 18% complete... Rolling back to snapshot: 20% complete... Rolling back to snapshot: 21% complete... Rolling back to snapshot: 23% complete... Rolling back to snapshot: 25% complete... Rolling back to snapshot: 26% complete... Rolling back to snapshot: 28% complete... Rolling back to snapshot: 29% complete... Rolling back to snapshot: 31% complete... Rolling back to snapshot: 32% complete... Rolling back to snapshot: 34% complete... Rolling back to snapshot: 35% complete... Rolling back to snapshot: 37% complete... Rolling back to snapshot: 39% complete... Rolling back to snapshot: 40% complete... Rolling back to snapshot: 42% complete... Rolling back to snapshot: 43% complete... Rolling back to snapshot: 45% complete... Rolling back to snapshot: 46% complete... Rolling back to snapshot: 48% complete... Rolling back to snapshot: 50% complete... Rolling back to snapshot: 51% complete... Rolling back to snapshot: 53% complete... Rolling back to snapshot: 54% complete... Rolling back to snapshot: 56% complete... Rolling back to snapshot: 57% complete... Rolling back to snapshot: 59% complete... Rolling back to snapshot: 60% complete... Rolling back to snapshot: 62% complete... Rolling back to snapshot: 64% complete... Rolling back to snapshot: 65% complete... Rolling back to snapshot: 67% complete... Rolling back to snapshot: 68% complete... Rolling back to snapshot: 70% complete... Rolling back to snapshot: 71% complete... Rolling back to snapshot: 73% complete... Rolling back to snapshot: 75% complete... Rolling back to snapshot: 76% complete... Rolling back to snapshot: 78% complete... Rolling back to snapshot: 79% complete... Rolling back to snapshot: 81% complete... Rolling back to snapshot: 82% complete... Rolling back to snapshot: 84% complete... Rolling back to snapshot: 85% complete... Rolling back to snapshot: 87% complete... Rolling back to snapshot: 89% complete... Rolling back to snapshot: 90% complete... Rolling back to snapshot: 92% complete... Rolling back to snapshot: 93% complete... Rolling back to snapshot: 95% complete... Rolling back to snapshot: 96% complete... Rolling back to snapshot: 98% complete... Rolling back to snapshot: 100% complete...done. 2023-10-18T16:02:20.404 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg1 2023-10-18T16:02:20.404 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 256 MiB' 2023-10-18T16:02:20.436 INFO:tasks.workunit.client.0.smithi082.stdout: size 256 MiB in 64 objects 2023-10-18T16:02:20.437 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg-diff1 2023-10-18T16:02:20.437 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 256 MiB' 2023-10-18T16:02:20.659 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:02:20.655+0000 7fd952af1700 0 --2- 172.21.15.82:0/3829541244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a40943cfb0 0x55a409435c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2023-10-18T16:02:20.872 INFO:tasks.workunit.client.0.smithi082.stdout: size 256 MiB in 64 objects 2023-10-18T16:02:20.872 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd export testimg1 /tmp/img1.snap1 2023-10-18T16:02:20.972 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-10-18T16:02:20.975 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd export testimg-diff1 /tmp/img-diff1.snap1 2023-10-18T16:02:21.067 INFO:tasks.workunit.client.0.smithi082.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-10-18T16:02:21.070 INFO:tasks.workunit.client.0.smithi082.stderr:+ cmp /tmp/img2 /tmp/img1.snap1 2023-10-18T16:02:21.219 INFO:tasks.workunit.client.0.smithi082.stderr:+ cmp /tmp/img2 /tmp/img-diff1.snap1 2023-10-18T16:02:21.367 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm testimg2 2023-10-18T16:02:21.463 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 48% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 87% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 98% complete...2023-10-18T16:02:21.460+0000 7f4c891bb700 0 -- 172.21.15.82:0/3805601165 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7f4c64008780 msgr2=0x7f4c6400abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-18T16:02:21.470 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:02:21.472 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm testimg3 2023-10-18T16:02:21.571 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete...2023-10-18T16:02:21.568+0000 7fe8735d6700 0 -- 172.21.15.82:0/751796063 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7fe84c006040 msgr2=0x7fe84c008450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:02:21.576 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:02:21.578 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create testimg2 -s 0 2023-10-18T16:02:21.605 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:02:21.601+0000 7f172cb86380 -1 librbd: Forced V1 image creation. 2023-10-18T16:02:21.612 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd cp testimg2 testimg3 2023-10-18T16:02:21.664 INFO:tasks.workunit.client.0.smithi082.stderr: Image copy: 100% complete...done. 2023-10-18T16:02:21.666 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd deep cp testimg2 testimg6 2023-10-18T16:02:21.722 INFO:tasks.workunit.client.0.smithi082.stderr: Image deep copy: 100% complete...done. 2023-10-18T16:02:21.723 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap rm --snap=snap1 testimg1 2023-10-18T16:02:21.980 INFO:tasks.workunit.client.0.smithi082.stderr: Removing snap: 100% complete...done. 2023-10-18T16:02:21.991 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap rm --snap=snap1 testimg-diff1 2023-10-18T16:02:22.989 INFO:tasks.workunit.client.0.smithi082.stderr: Removing snap: 100% complete...done. 2023-10-18T16:02:23.000 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info --snap=snap1 testimg1 2023-10-18T16:02:23.001 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'error setting snapshot context: (2) No such file or directory' 2023-10-18T16:02:23.033 INFO:tasks.workunit.client.0.smithi082.stdout:error setting snapshot context: (2) No such file or directory 2023-10-18T16:02:23.034 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info --snap=snap1 testimg-diff1 2023-10-18T16:02:23.034 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'error setting snapshot context: (2) No such file or directory' 2023-10-18T16:02:23.066 INFO:tasks.workunit.client.0.smithi082.stdout:error setting snapshot context: (2) No such file or directory 2023-10-18T16:02:23.066 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd sparsify testimg1 2023-10-18T16:02:23.136 INFO:tasks.workunit.client.0.smithi082.stderr: Image sparsify: 1% complete... Image sparsify: 3% complete... Image sparsify: 4% complete... Image sparsify: 6% complete... Image sparsify: 7% complete... Image sparsify: 9% complete... Image sparsify: 10% complete... Image sparsify: 12% complete... Image sparsify: 14% complete... Image sparsify: 15% complete... Image sparsify: 17% complete... Image sparsify: 18% complete... Image sparsify: 20% complete... Image sparsify: 21% complete... Image sparsify: 23% complete... Image sparsify: 25% complete... Image sparsify: 26% complete... Image sparsify: 28% complete... Image sparsify: 29% complete... Image sparsify: 31% complete... Image sparsify: 32% complete... Image sparsify: 34% complete... Image sparsify: 35% complete... Image sparsify: 37% complete... Image sparsify: 39% complete... Image sparsify: 40% complete... Image sparsify: 42% complete... Image sparsify: 43% complete... Image sparsify: 45% complete... Image sparsify: 46% complete... Image sparsify: 48% complete... Image sparsify: 50% complete... Image sparsify: 51% complete... Image sparsify: 53% complete... Image sparsify: 54% complete... Image sparsify: 56% complete... Image sparsify: 57% complete... Image sparsify: 59% complete... Image sparsify: 60% complete... Image sparsify: 62% complete... Image sparsify: 64% complete... Image sparsify: 65% complete... Image sparsify: 67% complete... Image sparsify: 68% complete... Image sparsify: 70% complete... Image sparsify: 71% complete... Image sparsify: 73% complete... Image sparsify: 75% complete... Image sparsify: 76% complete... Image sparsify: 78% complete... Image sparsify: 79% complete... Image sparsify: 81% complete... Image sparsify: 82% complete... Image sparsify: 84% complete... Image sparsify: 85% complete... Image sparsify: 87% complete... Image sparsify: 89% complete... Image sparsify: 90% complete... Image sparsify: 92% complete... Image sparsify: 93% complete... Image sparsify: 95% complete... Image sparsify: 96% complete... Image sparsify: 98% complete... Image sparsify: 100% complete...done. 2023-10-18T16:02:23.144 INFO:tasks.workunit.client.0.smithi082.stderr:+ remove_images 2023-10-18T16:02:23.144 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:23.287 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:23.562 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:23.675 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:24.104 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:25.113 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:25.230 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:25.366 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:25.507 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:25.648 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:25.726 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:25.800 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:25.876 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:25.951 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:26.026 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:26.102 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:26.180 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:26.254 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:26.331 INFO:tasks.workunit.client.0.smithi082.stderr:+ rm -f /tmp/img1 /tmp/img1.new /tmp/img2 /tmp/img2.new /tmp/img3 /tmp/img3.new /tmp/img-diff1.new /tmp/img-diff2.new /tmp/img-diff3.new /tmp/img1.snap1 /tmp/img1.snap1 /tmp/img-diff1.snap1 2023-10-18T16:02:26.448 INFO:tasks.workunit.client.0.smithi082.stderr:+ test_locking 2023-10-18T16:02:26.448 INFO:tasks.workunit.client.0.smithi082.stdout:testing locking... 2023-10-18T16:02:26.449 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'testing locking...' 2023-10-18T16:02:26.449 INFO:tasks.workunit.client.0.smithi082.stderr:+ remove_images 2023-10-18T16:02:26.449 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:26.523 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:26.597 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:26.672 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:26.748 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:26.824 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:26.899 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:26.973 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:27.049 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:27.125 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:27.201 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:27.277 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:27.349 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:27.423 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:27.498 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:27.572 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:27.647 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:27.722 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:27.797 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 -s 1 test1 2023-10-18T16:02:27.841 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd lock list test1 2023-10-18T16:02:27.842 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:27.842 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^0$' 2023-10-18T16:02:27.879 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:02:27.879 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd lock add test1 id 2023-10-18T16:02:27.918 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd lock list test1 2023-10-18T16:02:27.918 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep ' 1 ' 2023-10-18T16:02:27.952 INFO:tasks.workunit.client.0.smithi082.stdout:There is 1 exclusive lock on this image. 2023-10-18T16:02:27.953 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd lock list test1 2023-10-18T16:02:27.953 INFO:tasks.workunit.client.0.smithi082.stderr:++ tail -n 1 2023-10-18T16:02:27.954 INFO:tasks.workunit.client.0.smithi082.stderr:++ awk '{print $1;}' 2023-10-18T16:02:27.986 INFO:tasks.workunit.client.0.smithi082.stderr:+ LOCKER=client.8448 2023-10-18T16:02:27.987 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd lock remove test1 id client.8448 2023-10-18T16:02:29.003 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd lock list test1 2023-10-18T16:02:29.003 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:29.003 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^0$' 2023-10-18T16:02:29.039 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:02:29.039 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd lock add test1 id --shared tag 2023-10-18T16:02:29.082 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd lock list test1 2023-10-18T16:02:29.082 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep ' 1 ' 2023-10-18T16:02:29.115 INFO:tasks.workunit.client.0.smithi082.stdout:There is 1 shared lock on this image. 2023-10-18T16:02:29.116 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd lock add test1 id --shared tag 2023-10-18T16:02:29.157 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd lock list test1 2023-10-18T16:02:29.158 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep ' 2 ' 2023-10-18T16:02:29.190 INFO:tasks.workunit.client.0.smithi082.stdout:There are 2 shared locks on this image. 2023-10-18T16:02:29.191 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd lock add test1 id2 --shared tag 2023-10-18T16:02:29.232 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd lock list test1 2023-10-18T16:02:29.232 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep ' 3 ' 2023-10-18T16:02:29.265 INFO:tasks.workunit.client.0.smithi082.stdout:There are 3 shared locks on this image. 2023-10-18T16:02:29.266 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd lock list test1 2023-10-18T16:02:29.266 INFO:tasks.workunit.client.0.smithi082.stderr:+ tail -n 1 2023-10-18T16:02:29.266 INFO:tasks.workunit.client.0.smithi082.stderr:+ awk '{print $2, $1;}' 2023-10-18T16:02:29.266 INFO:tasks.workunit.client.0.smithi082.stderr:+ xargs rbd lock remove test1 2023-10-18T16:02:30.004 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info test1 2023-10-18T16:02:30.005 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -qE 'features:.*exclusive' 2023-10-18T16:02:30.041 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd lock list test1 2023-10-18T16:02:30.075 INFO:tasks.workunit.client.0.smithi082.stderr:+ '[' -n 'There are 2 shared locks on this image. 2023-10-18T16:02:30.075 INFO:tasks.workunit.client.0.smithi082.stderr:Lock tag: tag 2023-10-18T16:02:30.075 INFO:tasks.workunit.client.0.smithi082.stderr:Locker ID Address 2023-10-18T16:02:30.075 INFO:tasks.workunit.client.0.smithi082.stderr:client.8463 id 172.21.15.82:0/2074372305 2023-10-18T16:02:30.076 INFO:tasks.workunit.client.0.smithi082.stderr:client.8469 id 172.21.15.82:0/1871836410' ']' 2023-10-18T16:02:30.077 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd lock list test1 2023-10-18T16:02:30.077 INFO:tasks.workunit.client.0.smithi082.stderr:+ tail -n 1 2023-10-18T16:02:30.077 INFO:tasks.workunit.client.0.smithi082.stderr:+ awk '{print $2, $1;}' 2023-10-18T16:02:30.077 INFO:tasks.workunit.client.0.smithi082.stderr:+ xargs rbd lock remove test1 2023-10-18T16:02:31.013 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd lock list test1 2023-10-18T16:02:31.050 INFO:tasks.workunit.client.0.smithi082.stderr:+ '[' -n 'There is 1 shared lock on this image. 2023-10-18T16:02:31.051 INFO:tasks.workunit.client.0.smithi082.stderr:Lock tag: tag 2023-10-18T16:02:31.051 INFO:tasks.workunit.client.0.smithi082.stderr:Locker ID Address 2023-10-18T16:02:31.051 INFO:tasks.workunit.client.0.smithi082.stderr:client.8463 id 172.21.15.82:0/2074372305' ']' 2023-10-18T16:02:31.051 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd lock list test1 2023-10-18T16:02:31.052 INFO:tasks.workunit.client.0.smithi082.stderr:+ tail -n 1 2023-10-18T16:02:31.052 INFO:tasks.workunit.client.0.smithi082.stderr:+ awk '{print $2, $1;}' 2023-10-18T16:02:31.052 INFO:tasks.workunit.client.0.smithi082.stderr:+ xargs rbd lock remove test1 2023-10-18T16:02:32.016 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd lock list test1 2023-10-18T16:02:32.055 INFO:tasks.workunit.client.0.smithi082.stderr:+ '[' -n '' ']' 2023-10-18T16:02:32.055 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test1 2023-10-18T16:02:32.154 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:02:32.157 INFO:tasks.workunit.client.0.smithi082.stderr:+ test_clone 2023-10-18T16:02:32.157 INFO:tasks.workunit.client.0.smithi082.stdout:testing clone... 2023-10-18T16:02:32.158 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'testing clone...' 2023-10-18T16:02:32.158 INFO:tasks.workunit.client.0.smithi082.stderr:+ remove_images 2023-10-18T16:02:32.158 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:32.231 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:32.305 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:32.378 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:32.452 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:32.527 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:32.601 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:32.675 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:32.749 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:32.824 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:32.899 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:32.973 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:33.047 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:33.121 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:33.195 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:33.269 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:33.343 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:33.418 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:33.492 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create test1 --image-format 2 -s 1 2023-10-18T16:02:33.534 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create test1@s1 2023-10-18T16:02:34.029 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:02:34.037 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap protect test1@s1 2023-10-18T16:02:34.076 INFO:tasks.workunit.client.0.smithi082.stderr:+ ceph osd pool create rbd2 8 2023-10-18T16:02:35.130 INFO:tasks.workunit.client.0.smithi082.stderr:pool 'rbd2' already exists 2023-10-18T16:02:35.139 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd pool init rbd2 2023-10-18T16:02:38.040 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone test1@s1 rbd2/clone 2023-10-18T16:02:38.304 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd -p rbd2 ls 2023-10-18T16:02:38.305 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep clone 2023-10-18T16:02:38.333 INFO:tasks.workunit.client.0.smithi082.stdout:clone 2023-10-18T16:02:38.334 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd -p rbd2 ls -l 2023-10-18T16:02:38.334 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep clone 2023-10-18T16:02:38.334 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test1@s1 2023-10-18T16:02:38.375 INFO:tasks.workunit.client.0.smithi082.stdout:clone 1 MiB rbd/test1@s1 2 2023-10-18T16:02:38.376 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:02:38.376 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -v clone 2023-10-18T16:02:38.400 INFO:tasks.workunit.client.0.smithi082.stdout:test1 2023-10-18T16:02:38.401 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd flatten rbd2/clone 2023-10-18T16:02:38.443 INFO:tasks.workunit.client.0.smithi082.stderr: Image flatten: 100% complete...done. 2023-10-18T16:02:38.449 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create rbd2/clone@s1 2023-10-18T16:02:39.040 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:02:39.046 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap protect rbd2/clone@s1 2023-10-18T16:02:39.083 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone rbd2/clone@s1 clone2 2023-10-18T16:02:39.150 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:02:39.150 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep clone2 2023-10-18T16:02:39.175 INFO:tasks.workunit.client.0.smithi082.stdout:clone2 2023-10-18T16:02:39.176 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls -l 2023-10-18T16:02:39.176 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep clone2 2023-10-18T16:02:39.176 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep rbd2/clone@s1 2023-10-18T16:02:39.214 INFO:tasks.workunit.client.0.smithi082.stdout:clone2 1 MiB rbd2/clone@s1 2 2023-10-18T16:02:39.215 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd -p rbd2 ls 2023-10-18T16:02:39.215 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -v clone2 2023-10-18T16:02:39.242 INFO:tasks.workunit.client.0.smithi082.stdout:clone 2023-10-18T16:02:39.242 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm clone2 2023-10-18T16:02:39.341 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:02:39.344 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap unprotect rbd2/clone@s1 2023-10-18T16:02:39.381 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap rm rbd2/clone@s1 2023-10-18T16:02:40.043 INFO:tasks.workunit.client.0.smithi082.stderr: Removing snap: 100% complete...done. 2023-10-18T16:02:40.052 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm rbd2/clone 2023-10-18T16:02:40.137 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:02:40.134+0000 7f4276463700 0 -- 172.21.15.82:0/490792476 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x559c45fb24b0 msgr2=0x559c46056d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:02:40.139 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:02:40.142 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap unprotect test1@s1 2023-10-18T16:02:40.183 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap rm test1@s1 2023-10-18T16:02:41.056 INFO:tasks.workunit.client.0.smithi082.stderr: Removing snap: 100% complete...done. 2023-10-18T16:02:41.070 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test1 2023-10-18T16:02:41.175 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:02:41.178 INFO:tasks.workunit.client.0.smithi082.stderr:+ ceph osd pool rm rbd2 rbd2 --yes-i-really-really-mean-it 2023-10-18T16:02:42.158 INFO:tasks.workunit.client.0.smithi082.stderr:pool 'rbd2' does not exist 2023-10-18T16:02:42.168 INFO:tasks.workunit.client.0.smithi082.stderr:+ test_trash 2023-10-18T16:02:42.168 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'testing trash...' 2023-10-18T16:02:42.168 INFO:tasks.workunit.client.0.smithi082.stdout:testing trash... 2023-10-18T16:02:42.169 INFO:tasks.workunit.client.0.smithi082.stderr:+ remove_images 2023-10-18T16:02:42.169 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:42.247 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:42.321 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:42.396 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:42.470 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:42.547 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:42.621 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:42.695 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:42.771 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:42.845 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:42.920 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:42.995 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:43.071 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:43.146 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:43.221 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:43.296 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:43.572 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:43.649 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:43.725 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 -s 1 test1 2023-10-18T16:02:43.766 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 -s 1 test2 2023-10-18T16:02:43.813 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:02:43.814 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test1 2023-10-18T16:02:43.838 INFO:tasks.workunit.client.0.smithi082.stdout:test1 2023-10-18T16:02:43.839 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:02:43.839 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test2 2023-10-18T16:02:43.864 INFO:tasks.workunit.client.0.smithi082.stdout:test2 2023-10-18T16:02:43.865 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:02:43.865 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:43.866 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 2 2023-10-18T16:02:43.890 INFO:tasks.workunit.client.0.smithi082.stdout:2 2023-10-18T16:02:43.891 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls -l 2023-10-18T16:02:43.891 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'test1.*2.*' 2023-10-18T16:02:43.925 INFO:tasks.workunit.client.0.smithi082.stdout:test1 1 MiB 2 2023-10-18T16:02:43.925 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls -l 2023-10-18T16:02:43.925 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'test2.*2.*' 2023-10-18T16:02:43.959 INFO:tasks.workunit.client.0.smithi082.stdout:test2 1 MiB 2 2023-10-18T16:02:43.959 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv test1 2023-10-18T16:02:44.017 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:02:44.017 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test2 2023-10-18T16:02:44.042 INFO:tasks.workunit.client.0.smithi082.stdout:test2 2023-10-18T16:02:44.043 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:02:44.043 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:44.043 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 1 2023-10-18T16:02:44.068 INFO:tasks.workunit.client.0.smithi082.stdout:1 2023-10-18T16:02:44.069 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls -l 2023-10-18T16:02:44.069 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'test2.*2.*' 2023-10-18T16:02:44.101 INFO:tasks.workunit.client.0.smithi082.stdout:test2 1 MiB 2 2023-10-18T16:02:44.102 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:02:44.102 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test1 2023-10-18T16:02:44.127 INFO:tasks.workunit.client.0.smithi082.stdout:2261d23f8974 test1 2023-10-18T16:02:44.128 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:02:44.128 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:44.129 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 1 2023-10-18T16:02:44.153 INFO:tasks.workunit.client.0.smithi082.stdout:1 2023-10-18T16:02:44.154 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls -l 2023-10-18T16:02:44.154 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'test1.*USER.*' 2023-10-18T16:02:44.187 INFO:tasks.workunit.client.0.smithi082.stdout:2261d23f8974 test1 USER Wed Oct 18 16:02:44 2023 expired at Wed Oct 18 16:02:44 2023 2023-10-18T16:02:44.187 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls -l 2023-10-18T16:02:44.188 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -v 'protected until' 2023-10-18T16:02:44.219 INFO:tasks.workunit.client.0.smithi082.stdout:ID NAME SOURCE DELETED_AT STATUS PARENT 2023-10-18T16:02:44.220 INFO:tasks.workunit.client.0.smithi082.stdout:2261d23f8974 test1 USER Wed Oct 18 16:02:44 2023 expired at Wed Oct 18 16:02:44 2023 2023-10-18T16:02:44.220 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd trash ls 2023-10-18T16:02:44.221 INFO:tasks.workunit.client.0.smithi082.stderr:++ cut -d ' ' -f 1 2023-10-18T16:02:44.246 INFO:tasks.workunit.client.0.smithi082.stderr:+ ID=2261d23f8974 2023-10-18T16:02:44.246 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash rm 2261d23f8974 2023-10-18T16:02:44.302 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:02:44.304 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv test2 2023-10-18T16:02:44.359 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd trash ls 2023-10-18T16:02:44.360 INFO:tasks.workunit.client.0.smithi082.stderr:++ cut -d ' ' -f 1 2023-10-18T16:02:44.385 INFO:tasks.workunit.client.0.smithi082.stderr:+ ID=226477ac88 2023-10-18T16:02:44.386 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info --image-id 226477ac88 2023-10-18T16:02:44.386 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'rbd image '\''test2'\''' 2023-10-18T16:02:44.419 INFO:tasks.workunit.client.0.smithi082.stdout:rbd image 'test2': 2023-10-18T16:02:44.419 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash restore 226477ac88 2023-10-18T16:02:44.455 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:02:44.455 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test2 2023-10-18T16:02:44.480 INFO:tasks.workunit.client.0.smithi082.stdout:test2 2023-10-18T16:02:44.481 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:02:44.481 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:44.481 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 1 2023-10-18T16:02:44.506 INFO:tasks.workunit.client.0.smithi082.stdout:1 2023-10-18T16:02:44.506 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls -l 2023-10-18T16:02:44.507 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'test2.*2.*' 2023-10-18T16:02:44.539 INFO:tasks.workunit.client.0.smithi082.stdout:test2 1 MiB 2 2023-10-18T16:02:44.540 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv test2 --expires-at '3600 sec' 2023-10-18T16:02:44.595 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:02:44.595 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test2 2023-10-18T16:02:44.620 INFO:tasks.workunit.client.0.smithi082.stdout:226477ac88 test2 2023-10-18T16:02:44.621 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:02:44.621 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:44.621 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 1 2023-10-18T16:02:44.646 INFO:tasks.workunit.client.0.smithi082.stdout:1 2023-10-18T16:02:44.646 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls -l 2023-10-18T16:02:44.647 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'test2.*USER.*protected until' 2023-10-18T16:02:44.679 INFO:tasks.workunit.client.0.smithi082.stdout:226477ac88 test2 USER Wed Oct 18 16:02:44 2023 protected until Wed Oct 18 17:02:44 2023 2023-10-18T16:02:44.680 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash rm 226477ac88 2023-10-18T16:02:44.680 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'Deferment time has not expired' 2023-10-18T16:02:44.706 INFO:tasks.workunit.client.0.smithi082.stdout:Deferment time has not expired, please use --force if you really want to remove the image2023-10-18T16:02:44.700+0000 7fb38aefa380 -1 librbd::api::Trash: remove: error: deferment time has not expired. 2023-10-18T16:02:44.707 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash rm --image-id 226477ac88 --force 2023-10-18T16:02:44.763 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:02:44.765 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 -s 1 test1 2023-10-18T16:02:44.807 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create test1@snap1 2023-10-18T16:02:45.064 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:02:45.070 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap protect test1@snap1 2023-10-18T16:02:45.109 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv test1 2023-10-18T16:02:45.163 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:02:45.163 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test1 2023-10-18T16:02:45.188 INFO:tasks.workunit.client.0.smithi082.stdout:22bae98ed40e test1 2023-10-18T16:02:45.188 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:02:45.189 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:45.189 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 1 2023-10-18T16:02:45.214 INFO:tasks.workunit.client.0.smithi082.stdout:1 2023-10-18T16:02:45.214 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls -l 2023-10-18T16:02:45.215 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'test1.*USER.*' 2023-10-18T16:02:45.247 INFO:tasks.workunit.client.0.smithi082.stdout:22bae98ed40e test1 USER Wed Oct 18 16:02:45 2023 expired at Wed Oct 18 16:02:45 2023 2023-10-18T16:02:45.248 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls -l 2023-10-18T16:02:45.248 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -v 'protected until' 2023-10-18T16:02:45.281 INFO:tasks.workunit.client.0.smithi082.stdout:ID NAME SOURCE DELETED_AT STATUS PARENT 2023-10-18T16:02:45.281 INFO:tasks.workunit.client.0.smithi082.stdout:22bae98ed40e test1 USER Wed Oct 18 16:02:45 2023 expired at Wed Oct 18 16:02:45 2023 2023-10-18T16:02:45.282 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd trash ls 2023-10-18T16:02:45.282 INFO:tasks.workunit.client.0.smithi082.stderr:++ cut -d ' ' -f 1 2023-10-18T16:02:45.308 INFO:tasks.workunit.client.0.smithi082.stderr:+ ID=22bae98ed40e 2023-10-18T16:02:45.308 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap ls --image-id 22bae98ed40e 2023-10-18T16:02:45.308 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -v SNAPID 2023-10-18T16:02:45.309 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:45.309 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 1 2023-10-18T16:02:45.341 INFO:tasks.workunit.client.0.smithi082.stdout:1 2023-10-18T16:02:45.342 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap ls --image-id 22bae98ed40e 2023-10-18T16:02:45.342 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '.*snap1.*' 2023-10-18T16:02:45.376 INFO:tasks.workunit.client.0.smithi082.stdout: 34 snap1 1 MiB yes Wed Oct 18 16:02:45 2023 2023-10-18T16:02:45.376 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap unprotect --image-id 22bae98ed40e --snap snap1 2023-10-18T16:02:45.420 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap rm --image-id 22bae98ed40e --snap snap1 2023-10-18T16:02:46.066 INFO:tasks.workunit.client.0.smithi082.stderr: Removing snap: 100% complete...done. 2023-10-18T16:02:46.074 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap ls --image-id 22bae98ed40e 2023-10-18T16:02:46.075 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -v SNAPID 2023-10-18T16:02:46.075 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:46.075 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 0 2023-10-18T16:02:46.111 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:02:46.111 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash restore 22bae98ed40e 2023-10-18T16:02:46.146 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create test1@snap1 2023-10-18T16:02:47.076 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:02:47.085 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create test1@snap2 2023-10-18T16:02:47.946 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:02:47.957 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap ls --image-id 22bae98ed40e 2023-10-18T16:02:47.957 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -v SNAPID 2023-10-18T16:02:47.957 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:47.957 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 2 2023-10-18T16:02:47.994 INFO:tasks.workunit.client.0.smithi082.stdout:2 2023-10-18T16:02:47.995 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap purge --image-id 22bae98ed40e 2023-10-18T16:02:49.945 INFO:tasks.workunit.client.0.smithi082.stderr: Removing all snapshots: 50% complete... Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2023-10-18T16:02:49.957 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap ls --image-id 22bae98ed40e 2023-10-18T16:02:49.958 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -v SNAPID 2023-10-18T16:02:49.958 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:49.958 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 0 2023-10-18T16:02:49.994 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:02:49.995 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm --rbd_move_to_trash_on_remove=true --rbd_move_to_trash_on_remove_expire_seconds=3600 test1 2023-10-18T16:02:50.050 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:02:50.052 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:02:50.053 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test1 2023-10-18T16:02:50.078 INFO:tasks.workunit.client.0.smithi082.stdout:22bae98ed40e test1 2023-10-18T16:02:50.079 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:02:50.080 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:50.080 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 1 2023-10-18T16:02:50.104 INFO:tasks.workunit.client.0.smithi082.stdout:1 2023-10-18T16:02:50.105 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls -l 2023-10-18T16:02:50.105 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'test1.*USER.*protected until' 2023-10-18T16:02:50.137 INFO:tasks.workunit.client.0.smithi082.stdout:22bae98ed40e test1 USER Wed Oct 18 16:02:50 2023 protected until Wed Oct 18 17:02:50 2023 2023-10-18T16:02:50.138 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash rm 22bae98ed40e 2023-10-18T16:02:50.138 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'Deferment time has not expired' 2023-10-18T16:02:50.165 INFO:tasks.workunit.client.0.smithi082.stdout:Deferment time has not expired, please use --force if you 2023-10-18T16:02:50.159+0000 7f06a7969380 -1 librbd::api::Trash: remove: error: deferment time has not expired. 2023-10-18T16:02:50.165 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash rm --image-id 22bae98ed40e --force 2023-10-18T16:02:50.222 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:02:50.224 INFO:tasks.workunit.client.0.smithi082.stderr:+ remove_images 2023-10-18T16:02:50.224 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:50.299 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:50.575 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:50.651 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:50.726 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:50.802 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:50.878 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:50.952 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:51.028 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:51.103 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:51.178 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:51.253 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:51.328 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:51.403 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:51.478 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:51.553 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:51.628 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:51.704 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:51.780 INFO:tasks.workunit.client.0.smithi082.stderr:+ test_purge 2023-10-18T16:02:51.780 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'testing trash purge...' 2023-10-18T16:02:51.780 INFO:tasks.workunit.client.0.smithi082.stdout:testing trash purge... 2023-10-18T16:02:51.781 INFO:tasks.workunit.client.0.smithi082.stderr:+ remove_images 2023-10-18T16:02:51.781 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:51.855 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:51.930 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:52.004 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:52.081 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:52.357 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:52.432 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:52.506 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:52.582 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:52.657 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:52.733 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:52.807 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:52.882 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:52.958 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:53.033 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:53.108 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:53.183 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:53.260 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:02:53.336 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:02:53.337 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:53.337 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 0 2023-10-18T16:02:53.362 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:02:53.362 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash purge 2023-10-18T16:02:53.385 INFO:tasks.workunit.client.0.smithi082.stderr: Removing images: 100% complete...done. 2023-10-18T16:02:53.387 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 --size 256 testimg1 2023-10-18T16:02:53.429 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 --size 256 testimg2 2023-10-18T16:02:53.471 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg1 2023-10-18T16:02:53.523 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg2 2023-10-18T16:02:53.584 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:02:53.585 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:53.585 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 2 2023-10-18T16:02:53.610 INFO:tasks.workunit.client.0.smithi082.stdout:2 2023-10-18T16:02:53.610 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash purge 2023-10-18T16:02:53.901 INFO:tasks.workunit.client.0.smithi082.stderr: Removing images: 50% complete... Removing images: 100% complete... Removing images: 100% complete...done. 2023-10-18T16:02:53.904 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:02:53.904 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:53.904 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 0 2023-10-18T16:02:53.930 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:02:53.930 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 --size 256 testimg1 2023-10-18T16:02:53.971 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 --size 256 testimg2 2023-10-18T16:02:54.014 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg1 --expires-at '1 hour' 2023-10-18T16:02:54.077 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg2 --expires-at '3 hours' 2023-10-18T16:02:54.133 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:02:54.134 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:54.134 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 2 2023-10-18T16:02:54.159 INFO:tasks.workunit.client.0.smithi082.stdout:2 2023-10-18T16:02:54.159 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash purge 2023-10-18T16:02:54.183 INFO:tasks.workunit.client.0.smithi082.stderr: Removing images: 100% complete...done. 2023-10-18T16:02:54.185 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:02:54.186 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:54.186 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 2 2023-10-18T16:02:54.211 INFO:tasks.workunit.client.0.smithi082.stdout:2 2023-10-18T16:02:54.211 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash purge --expired-before 'now + 2 hours' 2023-10-18T16:02:54.273 INFO:tasks.workunit.client.0.smithi082.stderr: Removing images: 100% complete... Removing images: 100% complete...done. 2023-10-18T16:02:54.276 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:02:54.276 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:54.276 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 1 2023-10-18T16:02:54.301 INFO:tasks.workunit.client.0.smithi082.stdout:1 2023-10-18T16:02:54.302 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:02:54.302 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep testimg2 2023-10-18T16:02:54.327 INFO:tasks.workunit.client.0.smithi082.stdout:23ffd5b791b2 testimg2 2023-10-18T16:02:54.327 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash purge --expired-before 'now + 4 hours' 2023-10-18T16:02:54.388 INFO:tasks.workunit.client.0.smithi082.stderr: Removing images: 100% complete... Removing images: 100% complete...done. 2023-10-18T16:02:54.391 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:02:54.391 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:54.391 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 0 2023-10-18T16:02:54.416 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:02:54.416 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 --size 256 testimg1 2023-10-18T16:02:54.457 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create testimg1@snap 2023-10-18T16:02:54.951 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:02:54.960 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 --size 256 testimg2 2023-10-18T16:02:55.002 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 --size 256 testimg3 2023-10-18T16:02:55.046 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg1 2023-10-18T16:02:55.101 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg2 2023-10-18T16:02:55.162 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg3 2023-10-18T16:02:55.220 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:02:55.220 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:55.220 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 3 2023-10-18T16:02:55.245 INFO:tasks.workunit.client.0.smithi082.stdout:3 2023-10-18T16:02:55.245 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash purge 2023-10-18T16:02:55.246 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'some expired images could not be removed' 2023-10-18T16:02:55.385 INFO:tasks.workunit.client.0.smithi082.stdout:rbd: some expired images could not be removed 2023-10-18T16:02:55.386 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:02:55.386 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:55.387 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 1 2023-10-18T16:02:55.411 INFO:tasks.workunit.client.0.smithi082.stdout:1 2023-10-18T16:02:55.411 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:02:55.412 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep testimg1 2023-10-18T16:02:55.437 INFO:tasks.workunit.client.0.smithi082.stdout:242051ed5b06 testimg1 2023-10-18T16:02:55.437 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd trash ls 2023-10-18T16:02:55.438 INFO:tasks.workunit.client.0.smithi082.stderr:++ awk '{ print $1 }' 2023-10-18T16:02:55.463 INFO:tasks.workunit.client.0.smithi082.stderr:+ ID=242051ed5b06 2023-10-18T16:02:55.464 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap purge --image-id 242051ed5b06 2023-10-18T16:02:55.950 INFO:tasks.workunit.client.0.smithi082.stderr: Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2023-10-18T16:02:55.962 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash purge 2023-10-18T16:02:56.018 INFO:tasks.workunit.client.0.smithi082.stderr: Removing images: 100% complete... Removing images: 100% complete...done. 2023-10-18T16:02:56.021 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:02:56.021 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:56.021 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 0 2023-10-18T16:02:56.047 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:02:56.047 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 --size 256 testimg1 2023-10-18T16:02:56.089 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 --size 256 testimg2 2023-10-18T16:02:56.132 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create testimg2@snap 2023-10-18T16:02:56.961 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:02:56.969 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 --size 256 testimg3 2023-10-18T16:02:57.011 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg1 2023-10-18T16:02:57.081 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg2 2023-10-18T16:02:57.136 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg3 2023-10-18T16:02:57.198 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:02:57.198 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:57.198 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 3 2023-10-18T16:02:57.223 INFO:tasks.workunit.client.0.smithi082.stdout:3 2023-10-18T16:02:57.224 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash purge 2023-10-18T16:02:57.224 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'some expired images could not be removed' 2023-10-18T16:02:57.360 INFO:tasks.workunit.client.0.smithi082.stdout:rbd: some expired images could not be removed 2023-10-18T16:02:57.361 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:02:57.361 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:57.361 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 1 2023-10-18T16:02:57.386 INFO:tasks.workunit.client.0.smithi082.stdout:1 2023-10-18T16:02:57.386 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:02:57.387 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep testimg2 2023-10-18T16:02:57.412 INFO:tasks.workunit.client.0.smithi082.stdout:24503cad6539 testimg2 2023-10-18T16:02:57.413 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd trash ls 2023-10-18T16:02:57.413 INFO:tasks.workunit.client.0.smithi082.stderr:++ awk '{ print $1 }' 2023-10-18T16:02:57.438 INFO:tasks.workunit.client.0.smithi082.stderr:+ ID=24503cad6539 2023-10-18T16:02:57.439 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap purge --image-id 24503cad6539 2023-10-18T16:02:57.939 INFO:tasks.workunit.client.0.smithi082.stderr: Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2023-10-18T16:02:57.948 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash purge 2023-10-18T16:02:58.012 INFO:tasks.workunit.client.0.smithi082.stderr: Removing images: 100% complete... Removing images: 100% complete...done. 2023-10-18T16:02:58.014 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:02:58.014 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:58.014 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 0 2023-10-18T16:02:58.039 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:02:58.039 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 --size 256 testimg1 2023-10-18T16:02:58.082 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 --size 256 testimg2 2023-10-18T16:02:58.123 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 --size 256 testimg3 2023-10-18T16:02:58.166 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create testimg3@snap 2023-10-18T16:02:58.972 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:02:58.980 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg1 2023-10-18T16:02:59.040 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg2 2023-10-18T16:02:59.096 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg3 2023-10-18T16:02:59.151 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:02:59.152 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:59.152 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 3 2023-10-18T16:02:59.177 INFO:tasks.workunit.client.0.smithi082.stdout:3 2023-10-18T16:02:59.177 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash purge 2023-10-18T16:02:59.177 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'some expired images could not be removed' 2023-10-18T16:02:59.308 INFO:tasks.workunit.client.0.smithi082.stdout:rbd: some expired images could not be removed 2023-10-18T16:02:59.308 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:02:59.309 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:02:59.309 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 1 2023-10-18T16:02:59.333 INFO:tasks.workunit.client.0.smithi082.stdout:1 2023-10-18T16:02:59.334 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:02:59.334 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep testimg3 2023-10-18T16:02:59.359 INFO:tasks.workunit.client.0.smithi082.stdout:2480850ad9ed testimg3 2023-10-18T16:02:59.360 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd trash ls 2023-10-18T16:02:59.360 INFO:tasks.workunit.client.0.smithi082.stderr:++ awk '{ print $1 }' 2023-10-18T16:02:59.386 INFO:tasks.workunit.client.0.smithi082.stderr:+ ID=2480850ad9ed 2023-10-18T16:02:59.386 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap purge --image-id 2480850ad9ed 2023-10-18T16:02:59.965 INFO:tasks.workunit.client.0.smithi082.stderr: Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2023-10-18T16:02:59.980 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash purge 2023-10-18T16:03:00.041 INFO:tasks.workunit.client.0.smithi082.stderr: Removing images: 100% complete... Removing images: 100% complete...done. 2023-10-18T16:03:00.044 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:00.044 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:00.045 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 0 2023-10-18T16:03:00.069 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:03:00.070 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 --size 256 testimg1 2023-10-18T16:03:00.112 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create testimg1@snap 2023-10-18T16:03:00.977 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:03:00.986 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone --rbd-default-clone-format=2 testimg1@snap testimg2 2023-10-18T16:03:01.051 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap rm testimg1@snap 2023-10-18T16:03:01.093 INFO:tasks.workunit.client.0.smithi082.stderr: Removing snap: 100% complete...done. 2023-10-18T16:03:01.101 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 --size 256 testimg3 2023-10-18T16:03:01.145 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create testimg2@snap 2023-10-18T16:03:01.982 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:03:01.990 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone --rbd-default-clone-format=2 testimg2@snap testimg4 2023-10-18T16:03:02.065 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone --rbd-default-clone-format=2 testimg2@snap testimg5 2023-10-18T16:03:02.142 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap rm testimg2@snap 2023-10-18T16:03:02.189 INFO:tasks.workunit.client.0.smithi082.stderr: Removing snap: 100% complete...done. 2023-10-18T16:03:02.196 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create testimg4@snap 2023-10-18T16:03:02.988 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:03:02.997 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone --rbd-default-clone-format=2 testimg4@snap testimg6 2023-10-18T16:03:03.085 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap rm testimg4@snap 2023-10-18T16:03:03.142 INFO:tasks.workunit.client.0.smithi082.stderr: Removing snap: 100% complete...done. 2023-10-18T16:03:03.150 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg1 2023-10-18T16:03:03.207 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg2 2023-10-18T16:03:03.263 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg3 2023-10-18T16:03:03.323 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg4 2023-10-18T16:03:03.377 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:03.378 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:03.378 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 4 2023-10-18T16:03:03.403 INFO:tasks.workunit.client.0.smithi082.stdout:4 2023-10-18T16:03:03.403 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash purge 2023-10-18T16:03:03.404 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'some expired images could not be removed' 2023-10-18T16:03:03.601 INFO:tasks.workunit.client.0.smithi082.stdout:rbd: some expired images could not be removed 2023-10-18T16:03:03.602 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:03.602 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:03.603 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 3 2023-10-18T16:03:03.627 INFO:tasks.workunit.client.0.smithi082.stdout:3 2023-10-18T16:03:03.628 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:03.628 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep testimg1 2023-10-18T16:03:03.653 INFO:tasks.workunit.client.0.smithi082.stdout:24a77eb15ed8 testimg1 2023-10-18T16:03:03.654 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:03.654 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep testimg2 2023-10-18T16:03:03.679 INFO:tasks.workunit.client.0.smithi082.stdout:24ad91dd558f testimg2 2023-10-18T16:03:03.680 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:03.680 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep testimg4 2023-10-18T16:03:03.705 INFO:tasks.workunit.client.0.smithi082.stdout:24b9554e04cd testimg4 2023-10-18T16:03:03.705 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg6 2023-10-18T16:03:03.763 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:03.763 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:03.763 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 4 2023-10-18T16:03:03.788 INFO:tasks.workunit.client.0.smithi082.stdout:4 2023-10-18T16:03:03.789 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash purge 2023-10-18T16:03:03.790 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'some expired images could not be removed' 2023-10-18T16:03:04.143 INFO:tasks.workunit.client.0.smithi082.stdout:rbd: some expired images could not be removed 2023-10-18T16:03:04.144 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:04.144 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:04.144 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 2 2023-10-18T16:03:04.169 INFO:tasks.workunit.client.0.smithi082.stdout:2 2023-10-18T16:03:04.170 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:04.170 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep testimg1 2023-10-18T16:03:04.195 INFO:tasks.workunit.client.0.smithi082.stdout:24a77eb15ed8 testimg1 2023-10-18T16:03:04.195 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:04.196 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep testimg2 2023-10-18T16:03:04.221 INFO:tasks.workunit.client.0.smithi082.stdout:24ad91dd558f testimg2 2023-10-18T16:03:04.221 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg5 2023-10-18T16:03:04.278 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:04.279 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:04.279 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 3 2023-10-18T16:03:04.304 INFO:tasks.workunit.client.0.smithi082.stdout:3 2023-10-18T16:03:04.304 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash purge 2023-10-18T16:03:04.395 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:03:04.391+0000 7f0b64b11700 0 -- 172.21.15.82:0/892393588 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x7f0b4404fb30 msgr2=0x7f0b44051f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-18T16:03:04.406 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:03:04.402+0000 7f0b65312700 0 -- 172.21.15.82:0/892393588 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x5641436c7f90 msgr2=0x5641436ddd40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:03:06.049 INFO:tasks.workunit.client.0.smithi082.stderr: Removing images: 33% complete... Removing images: 66% complete... Removing images: 100% complete... Removing images: 100% complete...done. 2023-10-18T16:03:06.052 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:06.052 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:06.052 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 0 2023-10-18T16:03:06.081 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:03:06.082 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 --size 256 testimg1 2023-10-18T16:03:06.323 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create testimg1@snap 2023-10-18T16:03:06.999 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:03:07.011 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone --rbd-default-clone-format=2 testimg1@snap testimg2 2023-10-18T16:03:07.073 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap rm testimg1@snap 2023-10-18T16:03:07.113 INFO:tasks.workunit.client.0.smithi082.stderr: Removing snap: 100% complete...done. 2023-10-18T16:03:07.120 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 --size 256 testimg3 2023-10-18T16:03:07.164 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create testimg3@snap 2023-10-18T16:03:07.947 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:03:07.955 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create testimg2@snap 2023-10-18T16:03:08.999 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:03:09.008 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone --rbd-default-clone-format=2 testimg2@snap testimg4 2023-10-18T16:03:09.084 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone --rbd-default-clone-format=2 testimg2@snap testimg5 2023-10-18T16:03:09.160 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap rm testimg2@snap 2023-10-18T16:03:09.207 INFO:tasks.workunit.client.0.smithi082.stderr: Removing snap: 100% complete...done. 2023-10-18T16:03:09.214 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create testimg4@snap 2023-10-18T16:03:10.003 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:03:10.012 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone --rbd-default-clone-format=2 testimg4@snap testimg6 2023-10-18T16:03:10.097 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap rm testimg4@snap 2023-10-18T16:03:10.149 INFO:tasks.workunit.client.0.smithi082.stderr: Removing snap: 100% complete...done. 2023-10-18T16:03:10.156 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg1 2023-10-18T16:03:10.211 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg2 2023-10-18T16:03:10.270 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg3 2023-10-18T16:03:10.324 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg4 2023-10-18T16:03:10.380 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:10.381 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:10.381 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 4 2023-10-18T16:03:10.406 INFO:tasks.workunit.client.0.smithi082.stdout:4 2023-10-18T16:03:10.406 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash purge 2023-10-18T16:03:10.407 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'some expired images could not be removed' 2023-10-18T16:03:10.523 INFO:tasks.workunit.client.0.smithi082.stdout:rbd: some expired images could not be removed 2023-10-18T16:03:10.524 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:10.525 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:10.525 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 4 2023-10-18T16:03:10.549 INFO:tasks.workunit.client.0.smithi082.stdout:4 2023-10-18T16:03:10.549 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg6 2023-10-18T16:03:10.606 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:10.607 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:10.607 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 5 2023-10-18T16:03:10.631 INFO:tasks.workunit.client.0.smithi082.stdout:5 2023-10-18T16:03:10.632 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash purge 2023-10-18T16:03:10.632 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'some expired images could not be removed' 2023-10-18T16:03:11.201 INFO:tasks.workunit.client.0.smithi082.stdout:rbd: some expired images could not be removed 2023-10-18T16:03:11.202 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:11.202 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:11.202 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 3 2023-10-18T16:03:11.227 INFO:tasks.workunit.client.0.smithi082.stdout:3 2023-10-18T16:03:11.228 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:11.228 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep testimg1 2023-10-18T16:03:11.253 INFO:tasks.workunit.client.0.smithi082.stdout:250726784cc7 testimg1 2023-10-18T16:03:11.253 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:11.254 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep testimg2 2023-10-18T16:03:11.279 INFO:tasks.workunit.client.0.smithi082.stdout:250d89289778 testimg2 2023-10-18T16:03:11.279 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:11.280 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep testimg3 2023-10-18T16:03:11.305 INFO:tasks.workunit.client.0.smithi082.stdout:251374e87cf7 testimg3 2023-10-18T16:03:11.305 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg5 2023-10-18T16:03:11.364 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:11.364 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:11.365 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 4 2023-10-18T16:03:11.389 INFO:tasks.workunit.client.0.smithi082.stdout:4 2023-10-18T16:03:11.389 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash purge 2023-10-18T16:03:11.390 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'some expired images could not be removed' 2023-10-18T16:03:13.142 INFO:tasks.workunit.client.0.smithi082.stdout:rbd: some expired images could not be removed 2023-10-18T16:03:13.143 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:13.143 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:13.144 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 1 2023-10-18T16:03:13.168 INFO:tasks.workunit.client.0.smithi082.stdout:1 2023-10-18T16:03:13.169 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:13.169 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep testimg3 2023-10-18T16:03:13.194 INFO:tasks.workunit.client.0.smithi082.stdout:251374e87cf7 testimg3 2023-10-18T16:03:13.195 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd trash ls 2023-10-18T16:03:13.195 INFO:tasks.workunit.client.0.smithi082.stderr:++ awk '{ print $1 }' 2023-10-18T16:03:13.221 INFO:tasks.workunit.client.0.smithi082.stderr:+ ID=251374e87cf7 2023-10-18T16:03:13.221 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap purge --image-id 251374e87cf7 2023-10-18T16:03:14.008 INFO:tasks.workunit.client.0.smithi082.stderr: Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2023-10-18T16:03:14.024 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash purge 2023-10-18T16:03:14.082 INFO:tasks.workunit.client.0.smithi082.stderr: Removing images: 100% complete... Removing images: 100% complete...done. 2023-10-18T16:03:14.084 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:14.085 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:14.085 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 0 2023-10-18T16:03:14.110 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:03:14.111 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 --size 256 testimg1 2023-10-18T16:03:14.152 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create testimg1@snap 2023-10-18T16:03:15.018 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:03:15.027 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone --rbd-default-clone-format=2 testimg1@snap testimg2 2023-10-18T16:03:15.092 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap rm testimg1@snap 2023-10-18T16:03:15.133 INFO:tasks.workunit.client.0.smithi082.stderr: Removing snap: 100% complete...done. 2023-10-18T16:03:15.141 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 --size 256 testimg3 2023-10-18T16:03:15.183 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create testimg2@snap 2023-10-18T16:03:16.023 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:03:16.033 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone --rbd-default-clone-format=2 testimg2@snap testimg4 2023-10-18T16:03:16.108 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone --rbd-default-clone-format=2 testimg2@snap testimg5 2023-10-18T16:03:16.178 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap rm testimg2@snap 2023-10-18T16:03:16.225 INFO:tasks.workunit.client.0.smithi082.stderr: Removing snap: 100% complete...done. 2023-10-18T16:03:16.233 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create testimg4@snap 2023-10-18T16:03:17.025 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:03:17.035 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone --rbd-default-clone-format=2 testimg4@snap testimg6 2023-10-18T16:03:17.122 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap rm testimg4@snap 2023-10-18T16:03:17.175 INFO:tasks.workunit.client.0.smithi082.stderr: Removing snap: 100% complete...done. 2023-10-18T16:03:17.183 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm --rbd_move_parent_to_trash_on_remove=true testimg1 2023-10-18T16:03:17.255 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:03:17.258 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm --rbd_move_parent_to_trash_on_remove=true testimg2 2023-10-18T16:03:17.332 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:03:17.336 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg3 2023-10-18T16:03:17.396 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm --rbd_move_parent_to_trash_on_remove=true testimg4 2023-10-18T16:03:17.465 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:03:17.468 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:17.468 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:17.468 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 4 2023-10-18T16:03:17.493 INFO:tasks.workunit.client.0.smithi082.stdout:4 2023-10-18T16:03:17.494 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash purge 2023-10-18T16:03:17.494 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'some expired images could not be removed' 2023-10-18T16:03:17.696 INFO:tasks.workunit.client.0.smithi082.stdout:rbd: some expired images could not be removed 2023-10-18T16:03:17.697 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:17.698 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:17.698 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 3 2023-10-18T16:03:17.722 INFO:tasks.workunit.client.0.smithi082.stdout:3 2023-10-18T16:03:17.723 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:17.723 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep testimg1 2023-10-18T16:03:17.748 INFO:tasks.workunit.client.0.smithi082.stdout:25738e1fe332 testimg1 2023-10-18T16:03:17.749 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:17.749 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep testimg2 2023-10-18T16:03:17.774 INFO:tasks.workunit.client.0.smithi082.stdout:2579c1316aad testimg2 2023-10-18T16:03:17.775 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:17.775 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep testimg4 2023-10-18T16:03:17.800 INFO:tasks.workunit.client.0.smithi082.stdout:25843ce0eaf2 testimg4 2023-10-18T16:03:17.801 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg6 2023-10-18T16:03:17.857 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:17.857 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:17.857 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 4 2023-10-18T16:03:17.882 INFO:tasks.workunit.client.0.smithi082.stdout:4 2023-10-18T16:03:17.882 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash purge 2023-10-18T16:03:17.883 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'some expired images could not be removed' 2023-10-18T16:03:19.128 INFO:tasks.workunit.client.0.smithi082.stdout:rbd: some expired images could not be removed 2023-10-18T16:03:19.129 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:19.129 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:19.130 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 2 2023-10-18T16:03:19.154 INFO:tasks.workunit.client.0.smithi082.stdout:2 2023-10-18T16:03:19.155 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:19.155 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep testimg1 2023-10-18T16:03:19.181 INFO:tasks.workunit.client.0.smithi082.stdout:25738e1fe332 testimg1 2023-10-18T16:03:19.181 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:19.182 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep testimg2 2023-10-18T16:03:19.206 INFO:tasks.workunit.client.0.smithi082.stdout:2579c1316aad testimg2 2023-10-18T16:03:19.207 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg5 2023-10-18T16:03:19.265 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:19.265 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:19.265 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 3 2023-10-18T16:03:19.290 INFO:tasks.workunit.client.0.smithi082.stdout:3 2023-10-18T16:03:19.290 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash purge 2023-10-18T16:03:19.379 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:03:19.376+0000 7f1469759700 0 -- 172.21.15.82:0/1542620999 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x55bb0629d070 msgr2=0x55bb062b1d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:03:19.391 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:03:19.388+0000 7f1468f58700 0 -- 172.21.15.82:0/1542620999 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x7f144804fb30 msgr2=0x7f1448051f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:03:21.100 INFO:tasks.workunit.client.0.smithi082.stderr: Removing images: 33% complete... Removing images: 66% complete... Removing images: 100% complete... Removing images: 100% complete...done. 2023-10-18T16:03:21.104 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:21.104 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:21.104 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 0 2023-10-18T16:03:21.130 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:03:21.131 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 --size 256 testimg1 2023-10-18T16:03:21.177 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create testimg1@snap 2023-10-18T16:03:22.040 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:03:22.049 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone --rbd-default-clone-format=2 testimg1@snap testimg2 2023-10-18T16:03:22.113 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap rm testimg1@snap 2023-10-18T16:03:22.154 INFO:tasks.workunit.client.0.smithi082.stderr: Removing snap: 100% complete...done. 2023-10-18T16:03:22.163 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 --size 256 testimg3 2023-10-18T16:03:22.209 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create testimg3@snap 2023-10-18T16:03:23.044 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:03:23.055 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create testimg2@snap 2023-10-18T16:03:24.049 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:03:24.058 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone --rbd-default-clone-format=2 testimg2@snap testimg4 2023-10-18T16:03:24.135 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone --rbd-default-clone-format=2 testimg2@snap testimg5 2023-10-18T16:03:24.207 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap rm testimg2@snap 2023-10-18T16:03:24.253 INFO:tasks.workunit.client.0.smithi082.stderr: Removing snap: 100% complete...done. 2023-10-18T16:03:24.262 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create testimg4@snap 2023-10-18T16:03:25.052 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:03:25.063 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone --rbd-default-clone-format=2 testimg4@snap testimg6 2023-10-18T16:03:25.144 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap rm testimg4@snap 2023-10-18T16:03:25.196 INFO:tasks.workunit.client.0.smithi082.stderr: Removing snap: 100% complete...done. 2023-10-18T16:03:25.205 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm --rbd_move_parent_to_trash_on_remove=true testimg1 2023-10-18T16:03:25.279 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:03:25.282 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm --rbd_move_parent_to_trash_on_remove=true testimg2 2023-10-18T16:03:25.355 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:03:25.358 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg3 2023-10-18T16:03:25.415 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm --rbd_move_parent_to_trash_on_remove=true testimg4 2023-10-18T16:03:25.488 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:03:25.491 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:25.491 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:25.491 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 4 2023-10-18T16:03:25.516 INFO:tasks.workunit.client.0.smithi082.stdout:4 2023-10-18T16:03:25.517 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash purge 2023-10-18T16:03:25.517 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'some expired images could not be removed' 2023-10-18T16:03:25.645 INFO:tasks.workunit.client.0.smithi082.stdout:rbd: some expired images could not be removed 2023-10-18T16:03:25.646 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:25.646 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:25.646 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 4 2023-10-18T16:03:25.670 INFO:tasks.workunit.client.0.smithi082.stdout:4 2023-10-18T16:03:25.671 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg6 2023-10-18T16:03:25.732 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:25.732 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:25.732 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 5 2023-10-18T16:03:25.757 INFO:tasks.workunit.client.0.smithi082.stdout:5 2023-10-18T16:03:25.757 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash purge 2023-10-18T16:03:25.758 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'some expired images could not be removed' 2023-10-18T16:03:26.172 INFO:tasks.workunit.client.0.smithi082.stdout:rbd: some expired images could not be removed 2023-10-18T16:03:26.172 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:26.173 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:26.173 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 3 2023-10-18T16:03:26.197 INFO:tasks.workunit.client.0.smithi082.stdout:3 2023-10-18T16:03:26.198 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:26.198 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep testimg1 2023-10-18T16:03:26.223 INFO:tasks.workunit.client.0.smithi082.stdout:25d234d4cd24 testimg1 2023-10-18T16:03:26.224 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:26.224 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep testimg2 2023-10-18T16:03:26.249 INFO:tasks.workunit.client.0.smithi082.stdout:25d8a2c3716d testimg2 2023-10-18T16:03:26.250 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:26.250 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep testimg3 2023-10-18T16:03:26.275 INFO:tasks.workunit.client.0.smithi082.stdout:25de48f840a6 testimg3 2023-10-18T16:03:26.276 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash mv testimg5 2023-10-18T16:03:26.340 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:26.341 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:26.341 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 4 2023-10-18T16:03:26.366 INFO:tasks.workunit.client.0.smithi082.stdout:4 2023-10-18T16:03:26.366 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash purge 2023-10-18T16:03:26.367 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'some expired images could not be removed' 2023-10-18T16:03:28.043 INFO:tasks.workunit.client.0.smithi082.stdout:rbd: some expired images could not be removed 2023-10-18T16:03:28.044 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:28.044 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:28.044 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 1 2023-10-18T16:03:28.069 INFO:tasks.workunit.client.0.smithi082.stdout:1 2023-10-18T16:03:28.070 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:28.070 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep testimg3 2023-10-18T16:03:28.095 INFO:tasks.workunit.client.0.smithi082.stdout:25de48f840a6 testimg3 2023-10-18T16:03:28.096 INFO:tasks.workunit.client.0.smithi082.stderr:++ rbd trash ls 2023-10-18T16:03:28.096 INFO:tasks.workunit.client.0.smithi082.stderr:++ awk '{ print $1 }' 2023-10-18T16:03:28.122 INFO:tasks.workunit.client.0.smithi082.stderr:+ ID=25de48f840a6 2023-10-18T16:03:28.122 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap purge --image-id 25de48f840a6 2023-10-18T16:03:29.053 INFO:tasks.workunit.client.0.smithi082.stderr: Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2023-10-18T16:03:29.069 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash purge 2023-10-18T16:03:29.132 INFO:tasks.workunit.client.0.smithi082.stderr: Removing images: 100% complete... Removing images: 100% complete...done. 2023-10-18T16:03:29.135 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls 2023-10-18T16:03:29.135 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:29.135 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 0 2023-10-18T16:03:29.161 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:03:29.162 INFO:tasks.workunit.client.0.smithi082.stderr:+ test_deep_copy_clone 2023-10-18T16:03:29.162 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'testing deep copy clone...' 2023-10-18T16:03:29.162 INFO:tasks.workunit.client.0.smithi082.stdout:testing deep copy clone... 2023-10-18T16:03:29.163 INFO:tasks.workunit.client.0.smithi082.stderr:+ remove_images 2023-10-18T16:03:29.163 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:29.240 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:29.318 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:29.395 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:29.472 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:29.551 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:29.629 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:29.708 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:29.787 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:29.867 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:29.945 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:30.022 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:30.100 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:30.180 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:30.459 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:30.537 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:30.616 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:30.696 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:30.776 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create testimg1 --image-format 2 --size 256 2023-10-18T16:03:31.003 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:03:31.000+0000 7fcbc5242700 0 --2- 172.21.15.82:0/2342322022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0441e92f0 0x55d0441e21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2023-10-18T16:03:31.022 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create testimg1 --snap=snap1 2023-10-18T16:03:32.069 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:03:32.078 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap protect testimg1@snap1 2023-10-18T16:03:32.121 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone testimg1@snap1 testimg2 2023-10-18T16:03:32.192 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create testimg2@snap2 2023-10-18T16:03:33.075 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:03:33.085 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd deep copy testimg2 testimg3 2023-10-18T16:03:34.059 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:03:34.055+0000 7f1d0243c700 0 -- 172.21.15.82:0/3923731558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562ce2cbdbf0 msgr2=0x562ce2d5c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-18T16:03:34.086 INFO:tasks.workunit.client.0.smithi082.stderr: Image deep copy: 1% complete... Image deep copy: 3% complete... Image deep copy: 4% complete... Image deep copy: 6% complete... Image deep copy: 7% complete... Image deep copy: 9% complete... Image deep copy: 10% complete... Image deep copy: 12% complete... Image deep copy: 14% complete... Image deep copy: 15% complete... Image deep copy: 17% complete... Image deep copy: 18% complete... Image deep copy: 20% complete... Image deep copy: 21% complete... Image deep copy: 23% complete... Image deep copy: 25% complete... Image deep copy: 26% complete... Image deep copy: 28% complete... Image deep copy: 29% complete... Image deep copy: 31% complete... Image deep copy: 32% complete... Image deep copy: 34% complete... Image deep copy: 35% complete... Image deep copy: 37% complete... Image deep copy: 39% complete... Image deep copy: 40% complete... Image deep copy: 42% complete... Image deep copy: 43% complete... Image deep copy: 45% complete... Image deep copy: 46% complete... Image deep copy: 48% complete... Image deep copy: 50% complete... Image deep copy: 51% complete... Image deep copy: 53% complete... Image deep copy: 54% complete... Image deep copy: 56% complete... Image deep copy: 57% complete... Image deep copy: 59% complete... Image deep copy: 60% complete... Image deep copy: 62% complete... Image deep copy: 64% complete... Image deep copy: 65% complete... Image deep copy: 67% complete... Image deep copy: 68% complete... Image deep copy: 70% complete... Image deep copy: 71% complete... Image deep copy: 73% complete... Image deep copy: 75% complete... Image deep copy: 76% complete... Image deep copy: 78% complete... Image deep copy: 79% complete... Image deep copy: 81% complete... Image deep copy: 82% complete... Image deep copy: 84% complete... Image deep copy: 85% complete... Image deep copy: 87% complete... Image deep copy: 89% complete... Image deep copy: 90% complete... Image deep copy: 92% complete... Image deep copy: 93% complete... Image deep copy: 95% complete... Image deep copy: 96% complete... Image deep copy: 98% complete... Image deep copy: 100% complete... Image deep copy: 100% complete...done. 2023-10-18T16:03:34.090 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg3 2023-10-18T16:03:34.090 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 256 MiB' 2023-10-18T16:03:34.132 INFO:tasks.workunit.client.0.smithi082.stdout: size 256 MiB in 64 objects 2023-10-18T16:03:34.132 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg3 2023-10-18T16:03:34.133 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'parent: rbd/testimg1@snap1' 2023-10-18T16:03:34.171 INFO:tasks.workunit.client.0.smithi082.stdout: parent: rbd/testimg1@snap1 2023-10-18T16:03:34.172 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap ls testimg3 2023-10-18T16:03:34.172 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -v SNAPID 2023-10-18T16:03:34.172 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:34.172 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 1 2023-10-18T16:03:34.210 INFO:tasks.workunit.client.0.smithi082.stdout:1 2023-10-18T16:03:34.210 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap ls testimg3 2023-10-18T16:03:34.211 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '.*snap2.*' 2023-10-18T16:03:34.249 INFO:tasks.workunit.client.0.smithi082.stdout: 76 snap2 256 MiB Wed Oct 18 16:03:34 2023 2023-10-18T16:03:34.250 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg2 2023-10-18T16:03:34.250 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'features:.*deep-flatten' 2023-10-18T16:03:34.288 INFO:tasks.workunit.client.0.smithi082.stdout: features: layering, exclusive-lock, object-map, fast-diff, deep-flatten 2023-10-18T16:03:34.289 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg3 2023-10-18T16:03:34.289 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'features:.*deep-flatten' 2023-10-18T16:03:34.327 INFO:tasks.workunit.client.0.smithi082.stdout: features: layering, exclusive-lock, object-map, fast-diff, deep-flatten 2023-10-18T16:03:34.328 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd flatten testimg2 2023-10-18T16:03:34.377 INFO:tasks.workunit.client.0.smithi082.stderr: Image flatten: 1% complete... Image flatten: 3% complete... Image flatten: 4% complete... Image flatten: 6% complete... Image flatten: 7% complete... Image flatten: 9% complete... Image flatten: 10% complete... Image flatten: 12% complete... Image flatten: 14% complete... Image flatten: 15% complete... Image flatten: 17% complete... Image flatten: 18% complete... Image flatten: 20% complete... Image flatten: 21% complete... Image flatten: 23% complete... Image flatten: 25% complete... Image flatten: 26% complete... Image flatten: 28% complete... Image flatten: 29% complete... Image flatten: 31% complete... Image flatten: 32% complete... Image flatten: 34% complete... Image flatten: 35% complete... Image flatten: 37% complete... Image flatten: 39% complete... Image flatten: 40% complete... Image flatten: 42% complete... Image flatten: 43% complete... Image flatten: 45% complete... Image flatten: 46% complete... Image flatten: 48% complete... Image flatten: 50% complete... Image flatten: 51% complete... Image flatten: 53% complete... Image flatten: 54% complete... Image flatten: 56% complete... Image flatten: 57% complete... Image flatten: 59% complete... Image flatten: 60% complete... Image flatten: 62% complete... Image flatten: 64% complete... Image flatten: 65% complete... Image flatten: 67% complete... Image flatten: 68% complete... Image flatten: 70% complete... Image flatten: 71% complete... Image flatten: 73% complete... Image flatten: 75% complete... Image flatten: 76% complete... Image flatten: 78% complete... Image flatten: 79% complete... Image flatten: 81% complete... Image flatten: 82% complete... Image flatten: 84% complete... Image flatten: 85% complete... Image flatten: 87% complete... Image flatten: 89% complete... Image flatten: 90% complete... Image flatten: 92% complete... Image flatten: 93% complete... Image flatten: 95% complete... Image flatten: 96% complete... Image flatten: 98% complete... Image flatten: 100% complete...done. 2023-10-18T16:03:34.383 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd flatten testimg3 2023-10-18T16:03:34.433 INFO:tasks.workunit.client.0.smithi082.stderr: Image flatten: 1% complete... Image flatten: 3% complete... Image flatten: 4% complete... Image flatten: 6% complete... Image flatten: 7% complete... Image flatten: 9% complete... Image flatten: 10% complete... Image flatten: 12% complete... Image flatten: 14% complete... Image flatten: 15% complete... Image flatten: 17% complete... Image flatten: 18% complete... Image flatten: 20% complete... Image flatten: 21% complete... Image flatten: 23% complete... Image flatten: 25% complete... Image flatten: 26% complete... Image flatten: 28% complete... Image flatten: 29% complete... Image flatten: 31% complete... Image flatten: 32% complete... Image flatten: 34% complete... Image flatten: 35% complete... Image flatten: 37% complete... Image flatten: 39% complete... Image flatten: 40% complete... Image flatten: 42% complete... Image flatten: 43% complete... Image flatten: 45% complete... Image flatten: 46% complete... Image flatten: 48% complete... Image flatten: 50% complete... Image flatten: 51% complete... Image flatten: 53% complete... Image flatten: 54% complete... Image flatten: 56% complete... Image flatten: 57% complete... Image flatten: 59% complete... Image flatten: 60% complete... Image flatten: 62% complete... Image flatten: 64% complete... Image flatten: 65% complete... Image flatten: 67% complete... Image flatten: 68% complete... Image flatten: 70% complete... Image flatten: 71% complete... Image flatten: 73% complete... Image flatten: 75% complete... Image flatten: 76% complete... Image flatten: 78% complete... Image flatten: 79% complete... Image flatten: 81% complete... Image flatten: 82% complete... Image flatten: 84% complete... Image flatten: 85% complete... Image flatten: 87% complete... Image flatten: 89% complete... Image flatten: 90% complete... Image flatten: 92% complete... Image flatten: 93% complete... Image flatten: 95% complete... Image flatten: 96% complete... Image flatten: 98% complete... Image flatten: 100% complete...done. 2023-10-18T16:03:34.440 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap unprotect testimg1@snap1 2023-10-18T16:03:34.482 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap purge testimg2 2023-10-18T16:03:35.074 INFO:tasks.workunit.client.0.smithi082.stderr: Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2023-10-18T16:03:35.088 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap purge testimg3 2023-10-18T16:03:36.074 INFO:tasks.workunit.client.0.smithi082.stderr: Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2023-10-18T16:03:36.087 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm testimg2 2023-10-18T16:03:36.201 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 48% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 87% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 98% complete... Removing image: 100% complete...done. 2023-10-18T16:03:36.202 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm testimg3 2023-10-18T16:03:36.312 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 48% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 87% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 98% complete... Removing image: 100% complete...done. 2023-10-18T16:03:36.315 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap protect testimg1@snap1 2023-10-18T16:03:36.356 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone testimg1@snap1 testimg2 2023-10-18T16:03:36.425 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create testimg2@snap2 2023-10-18T16:03:37.085 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:03:37.095 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd deep copy --flatten testimg2 testimg3 2023-10-18T16:03:38.064 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:03:38.060+0000 7f254aba6700 0 -- 172.21.15.82:0/2383298641 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x561e94176a30 msgr2=0x561e941cfdf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:03:38.118 INFO:tasks.workunit.client.0.smithi082.stderr: Image deep copy: 1% complete...2023-10-18T16:03:38.113+0000 7f254aba6700 0 -- 172.21.15.82:0/2383298641 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x7f252c0371e0 msgr2=0x7f252c0395d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:03:38.697 INFO:tasks.workunit.client.0.smithi082.stderr: Image deep copy: 3% complete... Image deep copy: 4% complete... Image deep copy: 6% complete... Image deep copy: 7% complete... Image deep copy: 9% complete... Image deep copy: 10% complete... Image deep copy: 12% complete... Image deep copy: 14% complete... Image deep copy: 15% complete... Image deep copy: 17% complete... Image deep copy: 18% complete... Image deep copy: 20% complete... Image deep copy: 21% complete... Image deep copy: 23% complete... Image deep copy: 25% complete... Image deep copy: 26% complete... Image deep copy: 28% complete... Image deep copy: 29% complete... Image deep copy: 31% complete... Image deep copy: 32% complete... Image deep copy: 34% complete... Image deep copy: 35% complete... Image deep copy: 37% complete... Image deep copy: 39% complete... Image deep copy: 40% complete... Image deep copy: 42% complete... Image deep copy: 43% complete... Image deep copy: 45% complete... Image deep copy: 46% complete... Image deep copy: 48% complete... Image deep copy: 50% complete... Image deep copy: 51% complete... Image deep copy: 53% complete... Image deep copy: 54% complete... Image deep copy: 56% complete... Image deep copy: 57% complete... Image deep copy: 59% complete... Image deep copy: 60% complete... Image deep copy: 62% complete... Image deep copy: 64% complete... Image deep copy: 65% complete... Image deep copy: 67% complete... Image deep copy: 68% complete... Image deep copy: 70% complete... Image deep copy: 71% complete... Image deep copy: 73% complete... Image deep copy: 75% complete... Image deep copy: 76% complete... Image deep copy: 78% complete... Image deep copy: 79% complete... Image deep copy: 81% complete... Image deep copy: 82% complete... Image deep copy: 84% complete... Image deep copy: 85% complete... Image deep copy: 87% complete... Image deep copy: 89% complete... Image deep copy: 90% complete... Image deep copy: 92% complete... Image deep copy: 93% complete... Image deep copy: 95% complete... Image deep copy: 96% complete... Image deep copy: 98% complete... Image deep copy: 100% complete... Image deep copy: 100% complete...done. 2023-10-18T16:03:38.700 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg3 2023-10-18T16:03:38.700 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'size 256 MiB' 2023-10-18T16:03:38.733 INFO:tasks.workunit.client.0.smithi082.stdout: size 256 MiB in 64 objects 2023-10-18T16:03:38.733 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg3 2023-10-18T16:03:38.733 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -v parent: 2023-10-18T16:03:38.766 INFO:tasks.workunit.client.0.smithi082.stdout:rbd image 'testimg3': 2023-10-18T16:03:38.766 INFO:tasks.workunit.client.0.smithi082.stdout: size 256 MiB in 64 objects 2023-10-18T16:03:38.766 INFO:tasks.workunit.client.0.smithi082.stdout: order 22 (4 MiB objects) 2023-10-18T16:03:38.767 INFO:tasks.workunit.client.0.smithi082.stdout: snapshot_count: 1 2023-10-18T16:03:38.767 INFO:tasks.workunit.client.0.smithi082.stdout: id: 26ec26874742 2023-10-18T16:03:38.768 INFO:tasks.workunit.client.0.smithi082.stdout: block_name_prefix: rbd_data.26ec26874742 2023-10-18T16:03:38.768 INFO:tasks.workunit.client.0.smithi082.stdout: format: 2 2023-10-18T16:03:38.768 INFO:tasks.workunit.client.0.smithi082.stdout: features: layering, exclusive-lock, object-map, fast-diff, deep-flatten 2023-10-18T16:03:38.769 INFO:tasks.workunit.client.0.smithi082.stdout: op_features: 2023-10-18T16:03:38.769 INFO:tasks.workunit.client.0.smithi082.stdout: flags: 2023-10-18T16:03:38.769 INFO:tasks.workunit.client.0.smithi082.stdout: create_timestamp: Wed Oct 18 16:03:37 2023 2023-10-18T16:03:38.769 INFO:tasks.workunit.client.0.smithi082.stdout: access_timestamp: Wed Oct 18 16:03:37 2023 2023-10-18T16:03:38.770 INFO:tasks.workunit.client.0.smithi082.stdout: modify_timestamp: Wed Oct 18 16:03:37 2023 2023-10-18T16:03:38.771 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap ls testimg3 2023-10-18T16:03:38.771 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -v SNAPID 2023-10-18T16:03:38.772 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:38.772 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 1 2023-10-18T16:03:38.799 INFO:tasks.workunit.client.0.smithi082.stdout:1 2023-10-18T16:03:38.799 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap ls testimg3 2023-10-18T16:03:38.800 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '.*snap2.*' 2023-10-18T16:03:38.832 INFO:tasks.workunit.client.0.smithi082.stdout: 80 snap2 256 MiB Wed Oct 18 16:03:37 2023 2023-10-18T16:03:38.832 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd info testimg2 2023-10-18T16:03:38.833 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'features:.*deep-flatten' 2023-10-18T16:03:38.872 INFO:tasks.workunit.client.0.smithi082.stdout: features: layering, exclusive-lock, object-map, fast-diff, deep-flatten 2023-10-18T16:03:38.872 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd flatten testimg2 2023-10-18T16:03:38.921 INFO:tasks.workunit.client.0.smithi082.stderr: Image flatten: 1% complete... Image flatten: 3% complete... Image flatten: 4% complete... Image flatten: 6% complete... Image flatten: 7% complete... Image flatten: 9% complete... Image flatten: 10% complete... Image flatten: 12% complete... Image flatten: 14% complete... Image flatten: 15% complete... Image flatten: 17% complete... Image flatten: 18% complete... Image flatten: 20% complete... Image flatten: 21% complete... Image flatten: 23% complete... Image flatten: 25% complete... Image flatten: 26% complete... Image flatten: 28% complete... Image flatten: 29% complete... Image flatten: 31% complete... Image flatten: 32% complete... Image flatten: 34% complete... Image flatten: 35% complete... Image flatten: 37% complete... Image flatten: 39% complete... Image flatten: 40% complete... Image flatten: 42% complete... Image flatten: 43% complete... Image flatten: 45% complete... Image flatten: 46% complete... Image flatten: 48% complete... Image flatten: 50% complete... Image flatten: 51% complete... Image flatten: 53% complete... Image flatten: 54% complete... Image flatten: 56% complete... Image flatten: 57% complete... Image flatten: 59% complete... Image flatten: 60% complete... Image flatten: 62% complete... Image flatten: 64% complete... Image flatten: 65% complete... Image flatten: 67% complete... Image flatten: 68% complete... Image flatten: 70% complete... Image flatten: 71% complete... Image flatten: 73% complete... Image flatten: 75% complete... Image flatten: 76% complete... Image flatten: 78% complete... Image flatten: 79% complete... Image flatten: 81% complete... Image flatten: 82% complete... Image flatten: 84% complete... Image flatten: 85% complete... Image flatten: 87% complete... Image flatten: 89% complete... Image flatten: 90% complete... Image flatten: 92% complete... Image flatten: 93% complete... Image flatten: 95% complete... Image flatten: 96% complete... Image flatten: 98% complete... Image flatten: 100% complete...done. 2023-10-18T16:03:38.929 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap unprotect testimg1@snap1 2023-10-18T16:03:38.973 INFO:tasks.workunit.client.0.smithi082.stderr:+ remove_images 2023-10-18T16:03:38.973 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:39.219 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:40.231 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:41.304 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:41.383 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:41.462 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:41.540 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:41.620 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:41.698 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:41.783 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:41.862 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:41.940 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:42.018 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:42.097 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:42.176 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:42.258 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:42.538 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:42.619 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:42.699 INFO:tasks.workunit.client.0.smithi082.stderr:+ test_clone_v2 2023-10-18T16:03:42.699 INFO:tasks.workunit.client.0.smithi082.stdout:testing clone v2... 2023-10-18T16:03:42.700 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'testing clone v2...' 2023-10-18T16:03:42.700 INFO:tasks.workunit.client.0.smithi082.stderr:+ remove_images 2023-10-18T16:03:42.700 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:42.778 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:42.856 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:42.935 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:43.013 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:43.091 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:43.169 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:43.247 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:43.325 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:43.403 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:43.482 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:43.560 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:43.638 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:43.716 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:43.794 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:43.872 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:43.953 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:44.033 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:44.112 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 -s 1 test1 2023-10-18T16:03:44.159 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create test1@1 2023-10-18T16:03:45.107 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:03:45.117 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone --rbd-default-clone-format=1 test1@1 test2 2023-10-18T16:03:45.151 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:03:45.147+0000 7f48eaffd700 -1 librbd::image::CloneRequest: 0x55cd6dd7e800 validate_parent: parent snapshot must be protected 2023-10-18T16:03:45.152 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: clone error: (22) Invalid argument 2023-10-18T16:03:45.154 INFO:tasks.workunit.client.0.smithi082.stderr:+ true 2023-10-18T16:03:45.154 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone --rbd-default-clone-format=2 test1@1 test2 2023-10-18T16:03:45.216 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone --rbd-default-clone-format=2 test1@1 test3 2023-10-18T16:03:45.290 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap protect test1@1 2023-10-18T16:03:45.332 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone --rbd-default-clone-format=1 test1@1 test4 2023-10-18T16:03:45.406 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd children test1@1 2023-10-18T16:03:45.406 INFO:tasks.workunit.client.0.smithi082.stderr:+ sort 2023-10-18T16:03:45.407 INFO:tasks.workunit.client.0.smithi082.stderr:+ tr '\n' ' ' 2023-10-18T16:03:45.407 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -E 'test2.*test3.*test4' 2023-10-18T16:03:45.446 INFO:tasks.workunit.client.0.smithi082.stdout:rbd/test2 rbd/test3 rbd/test4 2023-10-18T16:03:45.447 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd children --descendants test1 2023-10-18T16:03:45.447 INFO:tasks.workunit.client.0.smithi082.stderr:+ sort 2023-10-18T16:03:45.448 INFO:tasks.workunit.client.0.smithi082.stderr:+ tr '\n' ' ' 2023-10-18T16:03:45.448 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -E 'test2.*test3.*test4' 2023-10-18T16:03:45.499 INFO:tasks.workunit.client.0.smithi082.stdout:rbd/test2 rbd/test3 rbd/test4 2023-10-18T16:03:45.500 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd remove test4 2023-10-18T16:03:45.615 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:03:45.618 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap unprotect test1@1 2023-10-18T16:03:45.660 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap remove test1@1 2023-10-18T16:03:45.702 INFO:tasks.workunit.client.0.smithi082.stderr: Removing snap: 100% complete...done. 2023-10-18T16:03:45.711 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap list --all test1 2023-10-18T16:03:45.711 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep -E 'trash \(1\) *$' 2023-10-18T16:03:45.746 INFO:tasks.workunit.client.0.smithi082.stdout: 84 ea97c0ce-c260-47a9-b894-4fbf00a83b38 1 MiB Wed Oct 18 16:03:45 2023 trash (1) 2023-10-18T16:03:45.746 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create test1@2 2023-10-18T16:03:46.114 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:03:46.123 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test1 2023-10-18T16:03:46.124 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'image has snapshots' 2023-10-18T16:03:46.178 INFO:tasks.workunit.client.0.smithi082.stdout:rbd: image has snapshots - these must be deleted with 'rbd snap purge' before the image can be removed. 2023-10-18T16:03:46.179 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap rm test1@2 2023-10-18T16:03:47.107 INFO:tasks.workunit.client.0.smithi082.stderr: Removing snap: 100% complete...done. 2023-10-18T16:03:47.122 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test1 2023-10-18T16:03:47.123 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'linked clones' 2023-10-18T16:03:47.182 INFO:tasks.workunit.client.0.smithi082.stdout:rbd: image has snapshots with linked clones - these must be deleted or flattened before the image can be removed. 2023-10-18T16:03:47.183 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test3 2023-10-18T16:03:47.286 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:03:47.283+0000 7fba85a98700 0 -- 172.21.15.82:0/3822948481 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x55ebdbac4290 msgr2=0x55ebdbb68d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:03:47.296 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:03:47.299 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test1 2023-10-18T16:03:47.299 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep 'linked clones' 2023-10-18T16:03:47.357 INFO:tasks.workunit.client.0.smithi082.stdout:rbd: image has snapshots with linked clones - these must be deleted or flattened before the image can be removed. 2023-10-18T16:03:47.358 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd flatten test2 2023-10-18T16:03:47.968 INFO:tasks.workunit.client.0.smithi082.stderr: Image flatten: 100% complete...done. 2023-10-18T16:03:47.977 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap list --all test1 2023-10-18T16:03:47.977 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:47.978 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^0$' 2023-10-18T16:03:48.011 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:03:48.011 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test1 2023-10-18T16:03:48.117 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:03:48.119 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test2 2023-10-18T16:03:48.224 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:03:48.227 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 -s 1 test1 2023-10-18T16:03:48.271 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create test1@1 2023-10-18T16:03:49.118 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:03:49.128 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap create test1@2 2023-10-18T16:03:50.126 INFO:tasks.workunit.client.0.smithi082.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-18T16:03:50.136 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone test1@1 test2 --rbd-default-clone-format 2 2023-10-18T16:03:50.212 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd clone test1@2 test3 --rbd-default-clone-format 2 2023-10-18T16:03:50.279 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap rm test1@1 2023-10-18T16:03:50.320 INFO:tasks.workunit.client.0.smithi082.stderr: Removing snap: 100% complete...done. 2023-10-18T16:03:50.329 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd snap rm test1@2 2023-10-18T16:03:50.371 INFO:tasks.workunit.client.0.smithi082.stderr: Removing snap: 100% complete...done. 2023-10-18T16:03:50.381 INFO:tasks.workunit.client.0.smithi082.stderr:+ expect_fail rbd rm test1 2023-10-18T16:03:50.382 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test1 2023-10-18T16:03:50.429 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:03:50.425+0000 7f9fd8c4b380 -1 librbd::api::Image: remove: image has snapshots - not removing 2023-10-18T16:03:50.429 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 0% complete...failed. 2023-10-18T16:03:50.434 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: image has snapshots with linked clones - these must be deleted or flattened before the image can be removed. 2023-10-18T16:03:50.437 INFO:tasks.workunit.client.0.smithi082.stderr:+ return 0 2023-10-18T16:03:50.438 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test1 --rbd-move-parent-to-trash-on-remove=true 2023-10-18T16:03:50.661 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:03:50.658+0000 7f9eae728700 0 --2- 172.21.15.82:0/2336895218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4f20e5220 0x55a4f217b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2023-10-18T16:03:50.766 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:03:50.769 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls -a 2023-10-18T16:03:50.769 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test1 2023-10-18T16:03:50.795 INFO:tasks.workunit.client.0.smithi082.stdout:28207200d161 test1 2023-10-18T16:03:50.796 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test2 2023-10-18T16:03:50.897 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:03:50.894+0000 7f69902b5700 0 -- 172.21.15.82:0/413185671 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x55e3d26c4520 msgr2=0x55e3d26d8b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-18T16:03:50.912 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:03:50.908+0000 7f698fab4700 0 -- 172.21.15.82:0/413185671 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7f696c0126e0 msgr2=0x7f696c010b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:03:51.155 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:03:51.157 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls -a 2023-10-18T16:03:51.158 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test1 2023-10-18T16:03:51.185 INFO:tasks.workunit.client.0.smithi082.stdout:28207200d161 test1 2023-10-18T16:03:51.185 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test3 2023-10-18T16:03:51.299 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:03:51.295+0000 7fe97f18f700 0 -- 172.21.15.82:0/2650133489 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x55abe16e14b0 msgr2=0x55abe1785d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:03:52.125 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:03:52.119+0000 7fe97f18f700 0 -- 172.21.15.82:0/2650133489 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x7fe9640367d0 msgr2=0x7fe964038c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:03:52.178 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:03:52.181 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd trash ls -a 2023-10-18T16:03:52.181 INFO:tasks.workunit.client.0.smithi082.stderr:+ expect_fail grep test1 2023-10-18T16:03:52.181 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test1 2023-10-18T16:03:52.208 INFO:tasks.workunit.client.0.smithi082.stderr:+ return 0 2023-10-18T16:03:52.209 INFO:tasks.workunit.client.0.smithi082.stderr:+ test_thick_provision 2023-10-18T16:03:52.209 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'testing thick provision...' 2023-10-18T16:03:52.209 INFO:tasks.workunit.client.0.smithi082.stdout:testing thick provision... 2023-10-18T16:03:52.210 INFO:tasks.workunit.client.0.smithi082.stderr:+ remove_images 2023-10-18T16:03:52.210 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:52.288 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:52.368 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:52.447 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:52.526 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:52.604 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:52.684 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:52.762 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:52.842 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:52.921 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:53.004 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:53.083 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:53.163 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:53.241 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:53.320 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:53.399 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:53.480 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:53.760 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:03:53.841 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 --thick-provision -s 64M test1 2023-10-18T16:03:54.173 INFO:tasks.workunit.client.0.smithi082.stderr: Thick provisioning: 6% complete... Thick provisioning: 12% complete... Thick provisioning: 18% complete... Thick provisioning: 25% complete... Thick provisioning: 31% complete... Thick provisioning: 37% complete... Thick provisioning: 43% complete... Thick provisioning: 50% complete... Thick provisioning: 56% complete... Thick provisioning: 62% complete... Thick provisioning: 68% complete... Thick provisioning: 75% complete... Thick provisioning: 81% complete... Thick provisioning: 87% complete... Thick provisioning: 93% complete... Thick provisioning: 100% complete... Thick provisioning: 100% complete...done. 2023-10-18T16:03:54.194 INFO:tasks.workunit.client.0.smithi082.stderr:+ count=0 2023-10-18T16:03:54.194 INFO:tasks.workunit.client.0.smithi082.stderr:+ ret= 2023-10-18T16:03:54.194 INFO:tasks.workunit.client.0.smithi082.stderr:+ '[' 0 -lt 10 ']' 2023-10-18T16:03:54.195 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd du 2023-10-18T16:03:54.195 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test1 2023-10-18T16:03:54.195 INFO:tasks.workunit.client.0.smithi082.stderr:+ tr -s ' ' 2023-10-18T16:03:54.195 INFO:tasks.workunit.client.0.smithi082.stderr:+ cut -d ' ' -f 4-5 2023-10-18T16:03:54.195 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^64 MiB' 2023-10-18T16:03:54.226 INFO:tasks.workunit.client.0.smithi082.stdout:64 MiB 2023-10-18T16:03:54.227 INFO:tasks.workunit.client.0.smithi082.stderr:+ ret=0 2023-10-18T16:03:54.227 INFO:tasks.workunit.client.0.smithi082.stderr:+ '[' 0 = 0 ']' 2023-10-18T16:03:54.227 INFO:tasks.workunit.client.0.smithi082.stderr:+ break 2023-10-18T16:03:54.227 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd du 2023-10-18T16:03:54.256 INFO:tasks.workunit.client.0.smithi082.stdout:NAME PROVISIONED USED 2023-10-18T16:03:54.256 INFO:tasks.workunit.client.0.smithi082.stdout:test1 64 MiB 64 MiB 2023-10-18T16:03:54.258 INFO:tasks.workunit.client.0.smithi082.stderr:+ '[' 0 '!=' 0 ']' 2023-10-18T16:03:54.259 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test1 2023-10-18T16:03:54.377 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 6% complete... Removing image: 12% complete... Removing image: 18% complete... Removing image: 25% complete... Removing image: 31% complete... Removing image: 37% complete... Removing image: 43% complete... Removing image: 50% complete... Removing image: 56% complete... Removing image: 62% complete... Removing image: 68% complete... Removing image: 75% complete... Removing image: 81% complete... Removing image: 87% complete... Removing image: 93% complete...2023-10-18T16:03:54.374+0000 7f4285db7700 0 -- 172.21.15.82:0/3445568991 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x55a2df5c6370 msgr2=0x55a2df666e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:03:54.411 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 100% complete...done. 2023-10-18T16:03:54.414 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:03:54.415 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test1 2023-10-18T16:03:54.415 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:03:54.415 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^0$' 2023-10-18T16:03:54.440 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:03:54.440 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 --thick-provision -s 4G test1 2023-10-18T16:03:54.916 INFO:tasks.workunit.client.0.smithi082.stderr: Thick provisioning: 1% complete... Thick provisioning: 2% complete... Thick provisioning: 3% complete... Thick provisioning: 4% complete... Thick provisioning: 5% complete...2023-10-18T16:03:54.912+0000 7fcc86841700 0 -- 172.21.15.82:0/1663619505 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x5607d3eb28f0 msgr2=0x5607d3ec56b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-18T16:03:55.021 INFO:tasks.workunit.client.0.smithi082.stderr: Thick provisioning: 6% complete...2023-10-18T16:03:55.017+0000 7fcc86841700 0 -- 172.21.15.82:0/1663619505 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7fcc68036700 msgr2=0x7fcc68038b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-18T16:04:05.436 INFO:tasks.workunit.client.0.smithi082.stderr: Thick provisioning: 7% complete... Thick provisioning: 8% complete... Thick provisioning: 9% complete... Thick provisioning: 10% complete... Thick provisioning: 11% complete... Thick provisioning: 12% complete... Thick provisioning: 13% complete... Thick provisioning: 14% complete... Thick provisioning: 15% complete... Thick provisioning: 16% complete... Thick provisioning: 17% complete... Thick provisioning: 18% complete... Thick provisioning: 19% complete... Thick provisioning: 20% complete... Thick provisioning: 21% complete... Thick provisioning: 22% complete... Thick provisioning: 23% complete... Thick provisioning: 24% complete... Thick provisioning: 25% complete... Thick provisioning: 26% complete... Thick provisioning: 27% complete... Thick provisioning: 28% complete... Thick provisioning: 29% complete... Thick provisioning: 30% complete... Thick provisioning: 31% complete... Thick provisioning: 32% complete... Thick provisioning: 33% complete... Thick provisioning: 34% complete... Thick provisioning: 35% complete... Thick provisioning: 36% complete... Thick provisioning: 37% complete... Thick provisioning: 38% complete... Thick provisioning: 39% complete... Thick provisioning: 40% complete... Thick provisioning: 41% complete... Thick provisioning: 42% complete... Thick provisioning: 43% complete... Thick provisioning: 44% complete... Thick provisioning: 45% complete... Thick provisioning: 46% complete... Thick provisioning: 47% complete... Thick provisioning: 48% complete... Thick provisioning: 49% complete... Thick provisioning: 50% complete... Thick provisioning: 51% complete... Thick provisioning: 52% complete... Thick provisioning: 53% complete... Thick provisioning: 54% complete... Thick provisioning: 55% complete... Thick provisioning: 56% complete... Thick provisioning: 57% complete... Thick provisioning: 58% complete... Thick provisioning: 59% complete... Thick provisioning: 60% complete... Thick provisioning: 61% complete... Thick provisioning: 62% complete... Thick provisioning: 63% complete... Thick provisioning: 64% complete... Thick provisioning: 65% complete... Thick provisioning: 66% complete... Thick provisioning: 67% complete... Thick provisioning: 68% complete... Thick provisioning: 69% complete... Thick provisioning: 70% complete... Thick provisioning: 71% complete... Thick provisioning: 72% complete... Thick provisioning: 73% complete... Thick provisioning: 74% complete... Thick provisioning: 75% complete... Thick provisioning: 76% complete... Thick provisioning: 77% complete... Thick provisioning: 78% complete... Thick provisioning: 79% complete... Thick provisioning: 80% complete... Thick provisioning: 81% complete... Thick provisioning: 82% complete... Thick provisioning: 83% complete... Thick provisioning: 84% complete... Thick provisioning: 85% complete... Thick provisioning: 86% complete... Thick provisioning: 87% complete... Thick provisioning: 88% complete... Thick provisioning: 89% complete... Thick provisioning: 90% complete... Thick provisioning: 91% complete... Thick provisioning: 92% complete... Thick provisioning: 93% complete... Thick provisioning: 94% complete... Thick provisioning: 95% complete... Thick provisioning: 96% complete... Thick provisioning: 97% complete... Thick provisioning: 98% complete... Thick provisioning: 99% complete... Thick provisioning: 100% complete... Thick provisioning: 100% complete...done. 2023-10-18T16:04:05.445 INFO:tasks.workunit.client.0.smithi082.stderr:+ count=0 2023-10-18T16:04:05.446 INFO:tasks.workunit.client.0.smithi082.stderr:+ ret= 2023-10-18T16:04:05.446 INFO:tasks.workunit.client.0.smithi082.stderr:+ '[' 0 -lt 10 ']' 2023-10-18T16:04:05.446 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd du 2023-10-18T16:04:05.447 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test1 2023-10-18T16:04:05.447 INFO:tasks.workunit.client.0.smithi082.stderr:+ tr -s ' ' 2023-10-18T16:04:05.447 INFO:tasks.workunit.client.0.smithi082.stderr:+ cut -d ' ' -f 4-5 2023-10-18T16:04:05.447 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^4 GiB' 2023-10-18T16:04:05.478 INFO:tasks.workunit.client.0.smithi082.stdout:4 GiB 2023-10-18T16:04:05.479 INFO:tasks.workunit.client.0.smithi082.stderr:+ ret=0 2023-10-18T16:04:05.479 INFO:tasks.workunit.client.0.smithi082.stderr:+ '[' 0 = 0 ']' 2023-10-18T16:04:05.479 INFO:tasks.workunit.client.0.smithi082.stderr:+ break 2023-10-18T16:04:05.479 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd du 2023-10-18T16:04:05.508 INFO:tasks.workunit.client.0.smithi082.stdout:NAME PROVISIONED USED 2023-10-18T16:04:05.508 INFO:tasks.workunit.client.0.smithi082.stdout:test1 4 GiB 4 GiB 2023-10-18T16:04:05.510 INFO:tasks.workunit.client.0.smithi082.stderr:+ '[' 0 '!=' 0 ']' 2023-10-18T16:04:05.511 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd rm test1 2023-10-18T16:04:05.618 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 1% complete... Removing image: 2% complete...2023-10-18T16:04:05.614+0000 7fe58d198700 0 -- 172.21.15.82:0/1962990317 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x559f2ab0b3c0 msgr2=0x559f2abacf40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:04:05.637 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 3% complete... Removing image: 4% complete...2023-10-18T16:04:05.633+0000 7fe58d198700 0 -- 172.21.15.82:0/1962990317 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x7fe5740366d0 msgr2=0x7fe574038b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-18T16:04:06.518 INFO:tasks.workunit.client.0.smithi082.stderr: Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2023-10-18T16:04:06.522 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd ls 2023-10-18T16:04:06.523 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test1 2023-10-18T16:04:06.523 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:04:06.523 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^0$' 2023-10-18T16:04:06.549 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:04:06.550 INFO:tasks.workunit.client.0.smithi082.stderr:+ test_namespace 2023-10-18T16:04:06.550 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'testing namespace...' 2023-10-18T16:04:06.550 INFO:tasks.workunit.client.0.smithi082.stdout:testing namespace... 2023-10-18T16:04:06.551 INFO:tasks.workunit.client.0.smithi082.stderr:+ remove_images 2023-10-18T16:04:06.551 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:04:06.832 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:04:06.911 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:04:06.991 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:04:07.070 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:04:07.150 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:04:07.235 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:04:07.315 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:04:07.395 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:04:07.474 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:04:07.554 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:04:07.634 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:04:07.714 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:04:07.792 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:04:07.873 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:04:07.951 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:04:08.030 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:04:08.111 INFO:tasks.workunit.client.0.smithi082.stderr:+ for img in $IMGS 2023-10-18T16:04:08.197 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd namespace ls 2023-10-18T16:04:08.197 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:04:08.197 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^0$' 2023-10-18T16:04:08.222 INFO:tasks.workunit.client.0.smithi082.stdout:0 2023-10-18T16:04:08.223 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd namespace create rbd/test1 2023-10-18T16:04:08.255 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd namespace create --pool rbd --namespace test2 2023-10-18T16:04:08.297 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd namespace create --namespace test3 2023-10-18T16:04:08.330 INFO:tasks.workunit.client.0.smithi082.stderr:+ expect_fail rbd namespace create rbd/test3 2023-10-18T16:04:08.331 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd namespace create rbd/test3 2023-10-18T16:04:08.359 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: failed to created namespace: (17) File exists 2023-10-18T16:04:08.359 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:04:08.356+0000 7f6f259fd380 -1 librbd::api::Namespace: create: failed to add namespace: (17) File exists 2023-10-18T16:04:08.361 INFO:tasks.workunit.client.0.smithi082.stderr:+ return 0 2023-10-18T16:04:08.362 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd namespace list 2023-10-18T16:04:08.362 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep test 2023-10-18T16:04:08.362 INFO:tasks.workunit.client.0.smithi082.stderr:+ wc -l 2023-10-18T16:04:08.362 INFO:tasks.workunit.client.0.smithi082.stderr:+ grep '^3$' 2023-10-18T16:04:08.387 INFO:tasks.workunit.client.0.smithi082.stdout:3 2023-10-18T16:04:08.387 INFO:tasks.workunit.client.0.smithi082.stderr:+ expect_fail rbd namespace remove --pool rbd missing 2023-10-18T16:04:08.388 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd namespace remove --pool rbd missing 2023-10-18T16:04:08.408 INFO:tasks.workunit.client.0.smithi082.stderr:rbd: namespace name was not specified 2023-10-18T16:04:08.409 INFO:tasks.workunit.client.0.smithi082.stderr:+ return 0 2023-10-18T16:04:08.409 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd create --image-format 2 --size 1G rbd/test1/image1 2023-10-18T16:04:08.458 INFO:tasks.workunit.client.0.smithi082.stderr:+ rbd bench --io-type write --io-pattern rand --io-total 32M --io-size 4K rbd/test1/image1 2023-10-18T16:04:08.507 INFO:tasks.workunit.client.0.smithi082.stdout:bench type write io_size 4096 io_threads 16 bytes 33554432 pattern random 2023-10-18T16:04:09.015 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:04:09.012+0000 7fd64ac56700 0 -- 172.21.15.82:0/3601686610 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x56278280c820 msgr2=0x56278280ec10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-18T16:04:09.344 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:04:09.341+0000 7fd64b457700 0 -- 172.21.15.82:0/3601686610 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x56278280c820 msgr2=0x7fd630038ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-18T16:04:09.857 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:04:09.854+0000 7fd64b457700 0 -- 172.21.15.82:0/3601686610 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x5627827aeb20 msgr2=0x7fd63004fc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-18T16:04:39.553 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:04:39.549+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.10581.0:1536 3.3 3:ef9b2db4:test1::rbd_data.295244d43cbc.0000000000000050:head [set-alloc-hint object_size 4194304 write_size 4194304,write 491520~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:39.939 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:04:39.935+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.10581.0:742 3.0 3:349e99d1:test1::rbd_data.295244d43cbc.0000000000000059:head [set-alloc-hint object_size 4194304 write_size 4194304,write 688128~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:40.555 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:04:40.551+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:40.962 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:04:40.958+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:41.555 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:04:41.551+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:41.931 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:04:41.927+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:42.586 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:04:42.583+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:42.912 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:04:42.908+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:43.600 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:04:43.596+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:43.954 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:04:43.950+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:44.576 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:04:44.572+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:44.940 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:04:44.936+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:45.604 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:04:45.600+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:45.905 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:04:45.901+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:46.647 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:04:46.643+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:46.933 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:04:46.929+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:47.695 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:04:47.691+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:47.961 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:04:47.957+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:48.723 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:04:48.719+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:48.924 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:04:48.920+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:49.728 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:04:49.724+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:49.886 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:04:49.882+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:50.707 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:04:50.703+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:50.900 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:04:50.896+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:51.747 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:04:51.743+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:51.945 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:04:51.941+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:52.728 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:04:52.724+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:52.931 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:04:52.927+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:53.703 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:04:53.699+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:53.910 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:04:53.906+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:54.705 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:04:54.701+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:54.911 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:04:54.907+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:55.719 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:04:55.715+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:55.868 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:04:55.864+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:56.743 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:04:56.739+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:56.849 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:04:56.846+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:57.740 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:04:57.737+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:57.801 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:04:57.798+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:58.742 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:04:58.739+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:58.835 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:04:58.831+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:59.702 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:04:59.699+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:04:59.885 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:04:59.882+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:00.670 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:00.667+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:00.907 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:00.904+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:01.703 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:01.700+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:01.877 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:01.874+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:02.669 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:02.666+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:02.866 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:02.863+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:03.716 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:03.713+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:03.882 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:03.879+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:04.673 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:04.670+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:04.933 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:04.930+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:05.716 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:05.713+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:05.974 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:05.971+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:06.702 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:06.699+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:06.963 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:06.960+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:07.662 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:07.659+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:07.957 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:07.954+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:08.691 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:08.688+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:08.974 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:08.971+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:09.675 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:09.672+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:09.932 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:09.929+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:10.657 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:10.654+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:10.937 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:10.933+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:11.608 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:11.606+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:11.926 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:11.923+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:12.659 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:12.656+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:12.897 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:12.895+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:13.658 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:13.655+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:13.904 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:13.901+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:14.633 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:14.630+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:14.946 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:14.943+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:15.626 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:15.622+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:15.900 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:15.896+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:16.582 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:16.578+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:16.875 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:16.871+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:17.597 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:17.593+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:17.905 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:17.902+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:18.608 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:18.604+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:18.884 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:18.880+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:19.638 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:19.634+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:19.922 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:19.918+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:20.590 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:20.586+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:20.919 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:20.915+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:21.564 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:21.560+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:21.878 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:21.874+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:22.612 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:22.607+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:22.830 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:22.826+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:23.574 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:23.570+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:23.830 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:23.826+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:24.586 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:24.582+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:24.858 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:24.854+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:25.542 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:25.538+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:25.835 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:25.831+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:26.555 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:26.551+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:26.803 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:26.799+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:27.592 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:27.588+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:27.786 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:27.782+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:28.558 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:28.554+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:28.804 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:28.800+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:29.564 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:29.560+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:29.815 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:29.812+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:30.523 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:30.519+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:30.810 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:30.806+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:31.498 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:31.494+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:31.813 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:31.810+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:32.453 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:32.449+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:32.810 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:32.806+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:33.426 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:33.422+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:33.772 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:33.768+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:34.459 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:34.456+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:34.754 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:34.750+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:35.487 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:35.483+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:35.722 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:35.718+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:36.458 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:36.454+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:36.724 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:36.720+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:37.464 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:37.460+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:37.675 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:37.672+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:38.470 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:38.466+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:38.709 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:38.705+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:39.494 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:39.490+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:39.756 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:39.752+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:40.532 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:40.528+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:40.737 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:40.733+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:41.546 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:41.542+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:41.739 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:41.736+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:42.557 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:42.553+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:42.724 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:42.719+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:43.530 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:43.526+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:43.767 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:43.763+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:44.516 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:44.512+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:44.734 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:44.730+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:45.487 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:45.483+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:45.715 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:45.711+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:46.523 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:46.520+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:46.745 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:46.741+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:47.527 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:47.523+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:47.750 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:47.746+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:48.514 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:48.510+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:48.797 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:48.794+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:49.511 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:49.508+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:49.755 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:49.752+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:50.557 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:50.554+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:50.737 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:50.734+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:51.605 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:51.602+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:51.714 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:51.712+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:52.655 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:52.652+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:52.746 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:52.743+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:53.653 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:53.650+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:53.725 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:53.722+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:54.635 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:54.632+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:54.721 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:54.717+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:55.594 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:55.591+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:55.703 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:55.700+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:56.586 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:56.583+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:56.736 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:56.733+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:57.598 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:57.595+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:57.773 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:57.770+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:58.588 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:58.585+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:58.759 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:58.756+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:59.556 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:05:59.553+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:05:59.773 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:05:59.770+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:00.572 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:00.569+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:00.769 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:00.766+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:01.610 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:01.607+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:01.781 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:01.778+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:02.611 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:02.609+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:02.756 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:02.753+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:03.618 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:03.616+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:03.794 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:03.791+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:04.608 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:04.605+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:04.774 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:04.771+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:05.589 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:05.586+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:05.771 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:05.768+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:06.552 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:06.549+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:06.783 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:06.779+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:07.563 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:07.559+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:07.748 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:07.744+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:08.575 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:08.571+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:08.797 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:08.793+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:09.537 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:09.533+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:09.771 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:09.767+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:10.569 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:10.565+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:10.761 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:10.756+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:11.570 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:11.566+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:11.738 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:11.734+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:12.605 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:12.601+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:12.724 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:12.720+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:13.578 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:13.574+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:13.727 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:13.723+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:14.627 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:14.623+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:14.732 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:14.728+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:15.593 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:15.589+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:15.764 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:15.760+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:16.601 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:16.597+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:16.719 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:16.715+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:17.561 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:17.557+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:17.753 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:17.749+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:18.601 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:18.597+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:18.744 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:18.740+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:19.565 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:19.561+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:19.777 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:19.773+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:20.593 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:20.589+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:20.778 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:20.775+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:21.621 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:21.617+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:21.793 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:21.790+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:22.628 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:22.624+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:22.765 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:22.762+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:23.601 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:23.597+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:23.720 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:23.716+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:24.556 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:24.553+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:24.700 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:24.697+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:25.590 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:25.587+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:25.688 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:25.685+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:26.629 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:26.625+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:26.643 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:26.639+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:27.628 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:27.624+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:27.663 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:27.659+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:28.630 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:28.626+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:28.705 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:28.701+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:29.616 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:29.612+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:29.737 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:29.733+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:30.575 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:30.571+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:30.751 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:30.747+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:31.583 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:31.580+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:31.752 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:31.748+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:32.565 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:32.561+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:32.790 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:32.786+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:33.521 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:33.517+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:33.776 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:33.773+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:34.498 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:34.494+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:34.756 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:34.752+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:35.459 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:35.455+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:35.795 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:35.791+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:36.416 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:36.413+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:36.841 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:36.837+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:37.378 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:37.374+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:37.843 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:37.839+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:38.402 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:38.398+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:38.855 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:38.851+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:39.436 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:39.432+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:39.901 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:39.897+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:40.446 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:40.443+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:40.921 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:40.918+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:41.466 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:41.463+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:41.929 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:41.926+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:42.430 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:42.427+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:42.963 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:42.960+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:43.413 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:43.410+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:43.957 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:43.952+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:44.406 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:44.403+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:44.982 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:44.979+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:45.432 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:45.429+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:46.000 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:45.997+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:46.398 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:46.395+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:47.049 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:47.046+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:47.441 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:47.438+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:48.035 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:48.032+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:48.417 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:48.414+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:49.000 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:48.997+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:49.450 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:49.447+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:50.015 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:50.012+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:50.428 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:50.425+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:51.018 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:51.015+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:51.394 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:51.391+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:52.064 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:52.061+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:52.379 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:52.377+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:53.091 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:53.088+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:53.407 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:53.404+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:54.133 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:54.130+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:54.395 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:54.392+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:55.184 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:55.181+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:55.438 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:55.435+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:56.157 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:56.155+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:56.419 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:56.417+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:57.171 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:57.168+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:57.466 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:57.463+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:58.198 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:58.194+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:58.476 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:58.472+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:59.177 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:06:59.173+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:06:59.447 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:06:59.443+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:00.153 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:00.149+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:00.476 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:00.472+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:01.168 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:01.164+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:01.497 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:01.493+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:02.212 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:02.208+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:02.468 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:02.465+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:03.212 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:03.208+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:03.495 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:03.491+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:04.213 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:04.209+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:04.474 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:04.470+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:05.203 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:05.199+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:05.435 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:05.431+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:06.188 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:06.184+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:06.463 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:06.459+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:07.215 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:07.212+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:07.448 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:07.444+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:08.246 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:08.242+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:08.461 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:08.458+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:09.225 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:09.222+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:09.470 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:09.466+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:10.208 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:10.204+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:10.512 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:10.508+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:11.220 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:11.216+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:11.503 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:11.499+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:12.182 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:12.178+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:12.485 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:12.481+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:13.153 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:13.149+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:13.524 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:13.520+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:14.159 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:14.155+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:14.517 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:14.513+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:15.206 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:15.202+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:15.529 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:15.525+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:16.186 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:16.182+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:16.538 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:16.534+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:17.207 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:17.203+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:17.513 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:17.510+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:18.248 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:18.244+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:18.506 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:18.502+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:19.209 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:19.206+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:19.474 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:19.470+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:20.163 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:20.159+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:20.425 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:20.421+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:21.141 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:21.137+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:21.406 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:21.402+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:22.174 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:22.170+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:22.411 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:22.407+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:23.206 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:23.202+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:23.368 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:23.364+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:24.180 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:24.176+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:24.343 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:24.339+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:25.192 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:25.188+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:25.316 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:25.312+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:26.147 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:26.143+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:26.297 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:26.278+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:27.130 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:27.126+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:27.322 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:27.318+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:28.141 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:28.138+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:28.295 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:28.291+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:29.101 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:29.097+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:29.310 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:29.306+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:30.072 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:30.068+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:30.326 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:30.322+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:31.092 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:31.088+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:31.288 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:31.284+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:32.105 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:32.102+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:32.286 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:32.284+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:33.122 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:33.119+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:33.297 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:33.294+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:34.111 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:34.108+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:34.268 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:34.265+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:35.131 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:35.128+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:35.272 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:35.269+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:36.112 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:36.109+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:36.234 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:36.231+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:37.136 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:37.133+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:37.273 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:37.270+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:38.170 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:38.167+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:38.252 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:38.249+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:39.217 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:39.214+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:39.291 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:39.288+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:40.219 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:40.216+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:40.312 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:40.309+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:41.253 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:41.250+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:41.325 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:41.322+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:42.271 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:42.268+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:42.285 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:42.282+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:43.243 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:43.240+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:43.263 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:43.260+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:44.247 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:44.244+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:44.300 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:44.297+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:45.245 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:45.242+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:45.262 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:45.260+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:46.229 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:46.226+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:46.277 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:46.274+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:47.235 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:47.232+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:47.242 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:47.239+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:48.222 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:48.219+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:48.274 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:48.271+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:49.184 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:49.181+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:49.312 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:49.309+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:50.216 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:50.212+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:50.269 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:50.265+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:51.235 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:51.231+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:51.315 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:51.311+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:52.265 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:52.261+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:52.295 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:52.291+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:53.235 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:53.230+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:53.341 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:53.337+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:54.246 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:54.242+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:54.387 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:54.383+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:55.256 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:55.252+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:55.339 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:55.335+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:56.241 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:56.237+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:56.384 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:56.380+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:57.200 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:57.196+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:57.344 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:57.340+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:58.195 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:58.191+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:58.313 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:58.309+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:59.201 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:07:59.197+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:07:59.360 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:07:59.356+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:00.186 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:00.182+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:00.387 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:00.384+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:01.197 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:01.193+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:01.393 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:01.389+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:02.242 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:02.239+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:02.420 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:02.417+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:03.257 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:03.253+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:03.466 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:03.462+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:04.298 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:04.294+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:04.474 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:04.470+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:05.322 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:05.319+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:05.439 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:05.435+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:06.275 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:06.271+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:06.396 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:06.393+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:07.299 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:07.295+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:07.376 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:07.372+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:08.348 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:08.344+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:08.393 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:08.389+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:09.379 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:09.375+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:09.389 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:09.385+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:10.373 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:10.369+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:10.409 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:10.405+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:11.338 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:11.334+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:11.431 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:11.427+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:12.318 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:12.314+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:12.406 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:12.402+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:13.272 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:13.269+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:13.405 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:13.401+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:14.315 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:14.311+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:14.449 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:14.445+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:15.351 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:15.347+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:15.472 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:15.468+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:16.341 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:16.337+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:16.486 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:16.483+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:17.319 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:17.315+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:17.526 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:17.523+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:18.362 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:18.358+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:18.501 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:18.497+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:19.345 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:19.341+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:19.536 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:19.532+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:20.395 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:20.392+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:20.528 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:20.524+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:21.421 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:21.417+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:21.563 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:21.559+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:22.376 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:22.372+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:22.592 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:22.588+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:23.396 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:23.393+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:23.635 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:23.631+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:24.357 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:24.354+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:24.603 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:24.599+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:25.337 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:25.334+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:25.610 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:25.607+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:26.311 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:26.308+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:26.621 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:26.618+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:27.323 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:27.320+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:27.621 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:27.618+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:28.280 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:28.277+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:28.595 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:28.592+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:29.299 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:29.296+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:29.610 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:29.608+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:30.277 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:30.274+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:30.638 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:30.635+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:31.265 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:31.262+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:31.648 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:31.645+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:32.266 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:32.264+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:32.665 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:32.663+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:33.242 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:33.239+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:33.696 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:33.693+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:34.242 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:34.239+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:34.663 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:34.660+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:35.253 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:35.250+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:35.623 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:35.620+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:36.256 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:36.253+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:36.621 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:36.618+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:37.252 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:37.249+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:37.578 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:37.575+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:38.213 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:38.210+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:38.571 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:38.568+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:39.216 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:39.214+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:39.551 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:39.548+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:40.194 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:40.191+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:40.553 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:40.550+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:41.243 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:41.240+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:41.558 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:41.555+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:42.264 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:42.260+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:42.598 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:42.594+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:43.233 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:43.229+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:43.569 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:43.565+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:44.218 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:44.215+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:44.560 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:44.556+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:45.236 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:45.232+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:45.516 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:45.512+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:46.200 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:46.197+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:46.536 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:46.532+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:47.215 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:47.211+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:47.501 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:47.496+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:48.254 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:48.251+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:48.505 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:48.501+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:49.305 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:49.301+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:49.531 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:49.527+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:50.350 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:50.347+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:50.501 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:50.497+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:51.385 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:51.381+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:51.532 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:51.528+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:52.346 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:52.342+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:52.537 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:52.533+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:53.316 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:53.313+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:53.567 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:53.563+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:54.327 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:54.323+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:54.605 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:54.601+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:55.377 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:55.373+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:55.617 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:55.613+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:56.388 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:56.384+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:56.620 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:56.617+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:57.415 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:57.411+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:57.615 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:57.611+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:58.380 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:58.376+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:58.579 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:58.575+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:59.360 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:08:59.356+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:08:59.575 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:08:59.571+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:00.352 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:00.348+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:00.545 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:00.541+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:01.379 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:01.375+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:01.568 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:01.565+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:02.353 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:02.349+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:02.591 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:02.587+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:03.355 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:03.351+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:03.577 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:03.573+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:04.385 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:04.381+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:04.614 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:04.610+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:05.391 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:05.387+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:05.576 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:05.572+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:06.364 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:06.360+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:06.586 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:06.582+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:07.373 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:07.369+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:07.593 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:07.589+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:08.402 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:08.399+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:08.575 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:08.571+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:09.427 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:09.423+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:09.552 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:09.549+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:10.431 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:10.427+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:10.597 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:10.593+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:11.405 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:11.401+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:11.627 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:11.623+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:12.359 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:12.355+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:12.582 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:12.578+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:13.394 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:13.391+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:13.590 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:13.587+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:14.413 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:14.409+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:14.577 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:14.573+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:15.373 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:15.369+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:15.557 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:15.553+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:16.419 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:16.416+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:16.571 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:16.568+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:17.409 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:17.406+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:17.536 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:17.533+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:18.437 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:18.434+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:18.497 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:18.494+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:19.400 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:19.398+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:19.540 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:19.538+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:20.397 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:20.394+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:20.570 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:20.568+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:21.434 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:21.431+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:21.614 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:21.611+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:22.468 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:22.465+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:22.596 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:22.594+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:23.428 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:23.425+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:23.563 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:23.560+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:24.425 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:24.422+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:24.569 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:24.566+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:25.434 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:25.432+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:25.552 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:25.550+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:26.434 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:26.431+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:26.550 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:26.547+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:27.423 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:27.420+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:27.561 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:27.558+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:28.421 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:28.418+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:28.579 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:28.576+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:29.401 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:29.398+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:29.553 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:29.550+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:30.390 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:30.387+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:30.529 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:30.526+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:31.399 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:31.396+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:31.505 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:31.502+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:32.357 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:32.354+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:32.539 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:32.536+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:33.365 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:33.363+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:33.503 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:33.500+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:34.368 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:34.364+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:34.523 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:34.519+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:35.330 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:35.326+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:35.522 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:35.518+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:36.286 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:36.282+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:36.492 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:36.488+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:37.333 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:37.329+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:37.448 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:37.445+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:38.377 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:38.373+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:38.482 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:38.479+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:39.388 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:39.384+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:39.511 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:39.507+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:40.343 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:40.339+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:40.522 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:40.518+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:41.373 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:41.369+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:41.570 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:41.567+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:42.362 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:42.359+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:42.565 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:42.561+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:43.364 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:43.360+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:43.538 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:43.534+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:44.387 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:44.383+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:44.500 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:44.496+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:45.341 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:45.337+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:45.541 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:45.537+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:46.310 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:46.306+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:46.514 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:46.510+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:47.342 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:47.338+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:47.485 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:47.481+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:48.389 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:48.385+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:48.481 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:48.478+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:49.383 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:49.379+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:49.490 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:49.487+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:50.419 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:50.415+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:50.470 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:50.466+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:51.435 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:51.431+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:51.435 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:51.431+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:52.428 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:52.424+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:52.467 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:52.463+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:53.458 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:53.455+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:53.515 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:53.511+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:54.498 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:54.493+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:54.535 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:54.531+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:55.453 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:55.449+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:55.490 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:55.486+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:56.448 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:56.444+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:56.471 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:56.467+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:57.488 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:57.484+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:57.493 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:57.489+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:58.491 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:58.487+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:58.502 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:58.498+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:59.495 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:09:59.492+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:09:59.503 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:09:59.499+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:00.524 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:00.520+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:00.535 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:00.532+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:01.486 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:01.482+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:01.506 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:01.502+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:02.485 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:02.479+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:02.530 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:02.526+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:03.460 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:03.456+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:03.518 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:03.514+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:04.492 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:04.488+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:04.530 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:04.526+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:05.444 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:05.440+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:05.563 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:05.559+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:06.461 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:06.457+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:06.534 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:06.530+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:07.426 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:07.422+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:07.545 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:07.542+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:08.409 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:08.406+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:08.531 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:08.528+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:09.414 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:09.411+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:09.574 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:09.572+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:10.464 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:10.461+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:10.605 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:10.602+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:11.453 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:11.450+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:11.651 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:11.648+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:12.455 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:12.452+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:12.628 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:12.626+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:13.462 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:13.459+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:13.674 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:13.672+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:14.423 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:14.420+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:14.697 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:14.694+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:15.392 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:15.390+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:15.730 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:15.727+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:16.437 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:16.434+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:16.700 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:16.697+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:17.431 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:17.428+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:17.734 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:17.731+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:18.412 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:18.409+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:18.721 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:18.718+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:19.421 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:19.418+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:19.753 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:19.750+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:20.427 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:20.424+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:20.782 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:20.780+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:21.462 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:21.460+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:21.811 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:21.808+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:22.471 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:22.468+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:22.773 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:22.770+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:23.483 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:23.480+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:23.730 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:23.727+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:24.531 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:24.528+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:24.685 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:24.682+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:25.570 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:25.568+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:25.665 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:25.661+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:26.573 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:26.569+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:26.623 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:26.620+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:27.601 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:27.597+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:27.609 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:27.605+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:28.598 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:28.595+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:28.650 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:28.647+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:29.643 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:29.639+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:29.692 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:29.689+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:30.684 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:30.681+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:30.713 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:30.709+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:31.709 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:31.705+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:31.735 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:31.731+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:32.683 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:32.679+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:32.734 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:32.730+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:33.638 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:33.634+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:33.690 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:33.686+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:34.677 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:34.673+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:34.714 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:34.710+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:35.721 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:35.717+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:35.758 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:35.754+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:36.706 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:36.703+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:36.760 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:36.757+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:37.698 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:37.692+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:37.751 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:37.747+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:38.658 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:38.654+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:38.787 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:38.783+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:39.681 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:39.677+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:39.777 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:39.773+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:40.709 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:40.705+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:40.790 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:40.787+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:41.749 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:41.745+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:41.799 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:41.795+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:42.773 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:42.769+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:42.773 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:42.769+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:43.780 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:43.776+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:43.800 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:43.797+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:44.776 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:44.773+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:44.780 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:44.776+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:45.745 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:45.741+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:45.795 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:45.791+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:46.712 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:46.708+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:46.757 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:46.754+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:47.693 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:47.689+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:47.786 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:47.782+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:48.734 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:48.731+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:48.821 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:48.817+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:49.688 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:49.684+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:49.820 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:49.816+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:50.692 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:50.688+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:50.851 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:50.847+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:51.723 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:51.720+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:51.893 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:51.889+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:52.711 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:52.707+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:52.922 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:52.918+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:53.732 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:53.728+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:53.907 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:53.903+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:54.686 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:54.682+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:54.943 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:54.939+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:55.726 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:55.723+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:55.953 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:55.949+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:56.756 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:56.753+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:56.952 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:56.948+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:57.709 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:57.706+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:57.939 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:57.936+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:58.756 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:58.753+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:58.898 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:58.894+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:59.786 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:10:59.783+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:10:59.945 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:10:59.942+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:00.769 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:00.766+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:00.958 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:00.956+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:01.723 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:01.720+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:01.957 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:01.954+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:02.678 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:02.674+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:02.908 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:02.905+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:03.647 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:03.644+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:03.926 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:03.923+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:04.637 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:04.634+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:04.911 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:04.908+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:05.601 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:05.598+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:05.887 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:05.884+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:06.641 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:06.638+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:06.886 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:06.883+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:07.623 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:07.620+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:07.914 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:07.912+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:08.659 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:08.656+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:08.893 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:08.891+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:09.643 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:09.640+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:09.888 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:09.885+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:10.680 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:10.677+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:10.934 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:10.932+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:11.713 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:11.710+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:11.973 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:11.968+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:12.734 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:12.731+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:12.996 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:12.994+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:13.707 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:13.704+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:13.959 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:13.956+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:14.728 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:14.725+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:15.007 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:15.004+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:15.692 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:15.688+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:16.004 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:16.002+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:16.666 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:16.663+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:17.013 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:17.010+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:17.679 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:17.675+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:17.996 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:17.992+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:18.682 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:18.678+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:19.038 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:19.035+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:19.732 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:19.728+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:20.017 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:20.013+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:20.688 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:20.685+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:20.993 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:20.989+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:21.719 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:21.715+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:21.954 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:21.951+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:22.675 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:22.671+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:22.978 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:22.974+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:23.671 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:23.668+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:23.935 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:23.932+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:24.697 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:24.693+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:24.980 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:24.976+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:25.653 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:25.649+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:25.961 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:25.957+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:26.656 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:26.652+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:26.993 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:26.989+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:27.673 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:27.669+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:28.018 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:28.013+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:28.712 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:28.708+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:28.971 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:28.967+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:29.730 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:29.726+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:29.990 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:29.987+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:30.717 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:30.713+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:31.022 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:31.019+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:31.695 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:31.691+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:31.990 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:31.987+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:32.683 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:32.679+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:33.015 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:33.009+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:33.683 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:33.679+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:34.023 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:34.019+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:34.732 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:34.728+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:35.006 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:35.002+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:35.764 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:35.761+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:36.011 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:36.007+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:36.721 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:36.717+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:37.005 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:37.002+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:37.679 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:37.675+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:38.007 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:38.003+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:38.708 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:38.704+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:38.967 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:38.964+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:39.686 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:39.683+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:39.989 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:39.985+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:40.717 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:40.713+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:40.985 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:40.981+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:41.732 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:41.728+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:41.980 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:41.976+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:42.733 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:42.729+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:42.945 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:42.941+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:43.770 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:43.766+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:43.924 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:43.920+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:44.749 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:44.746+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:44.954 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:44.950+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:45.796 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:45.792+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:45.989 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:45.985+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:46.813 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:46.810+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:46.941 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:46.938+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:47.825 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:47.821+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:47.971 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:47.967+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:48.875 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:48.871+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:48.923 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:48.919+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:49.887 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:49.883+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:49.966 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:49.963+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:50.872 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:50.869+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:51.008 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:51.005+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:51.866 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:51.863+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:52.019 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:52.016+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:52.901 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:52.898+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:53.003 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:53.000+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:53.866 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:53.863+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:54.019 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:54.016+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:54.839 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:54.836+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:54.978 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:54.976+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:55.840 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:55.837+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:55.946 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:55.944+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:56.805 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:56.802+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:56.907 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:56.904+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:57.814 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:57.812+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:57.931 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:57.928+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:58.856 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:58.854+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:58.934 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:58.927+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:59.862 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:11:59.860+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:11:59.949 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:11:59.946+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:00.821 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:00.818+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:00.940 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:00.938+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:01.861 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:01.858+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:01.923 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:01.920+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:02.881 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:02.878+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:02.918 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:02.915+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:03.846 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:03.843+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:03.871 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:03.868+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:04.837 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:04.834+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:04.903 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:04.900+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:05.800 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:05.797+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:05.874 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:05.871+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:06.808 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:06.805+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:06.910 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:06.907+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:07.824 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:07.821+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:07.931 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:07.928+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:08.799 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:08.796+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:08.901 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:08.898+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:09.771 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:09.767+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:09.892 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:09.889+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:10.755 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:10.751+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:10.940 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:10.936+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:11.789 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:11.785+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:11.955 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:11.952+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:12.741 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:12.737+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:12.968 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:12.965+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:13.711 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:13.707+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:13.943 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:13.940+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:14.733 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:14.729+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:14.906 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:14.902+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:15.702 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:15.698+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:15.890 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:15.886+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:16.675 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:16.671+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:16.927 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:16.923+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:17.680 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:17.676+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:17.934 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:17.931+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:18.670 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:18.666+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:18.976 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:18.973+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:19.680 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:19.676+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:19.980 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:19.976+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:20.658 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:20.655+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:20.972 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:20.969+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:21.637 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:21.633+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:21.995 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:21.991+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:22.685 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:22.682+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:23.043 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:23.040+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:23.715 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:23.711+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:24.090 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:24.086+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:24.721 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:24.717+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:25.081 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:25.075+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:25.735 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:25.731+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:26.106 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:26.102+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:26.722 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:26.718+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:27.129 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:27.125+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:27.739 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:27.735+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:28.092 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:28.088+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:28.741 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:28.737+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:29.097 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:29.093+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:29.779 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:29.775+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:30.125 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:30.122+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:30.736 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:30.733+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:31.105 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:31.101+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:31.730 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:31.727+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:32.095 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:32.091+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:32.717 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:32.713+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:33.139 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:33.135+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:33.701 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:33.697+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:34.115 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:34.111+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:34.678 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:34.674+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:35.122 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:35.118+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:35.714 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:35.710+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:36.107 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:36.103+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:36.756 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:36.752+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:37.148 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:37.144+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:37.760 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:37.757+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:38.171 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:38.167+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:38.783 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:38.779+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:39.164 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:39.161+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:39.737 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:39.733+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:40.205 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:40.201+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:40.743 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:40.739+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:41.228 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:41.224+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:41.747 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:41.743+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:42.208 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:42.204+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:42.698 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:42.695+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:43.194 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:43.192+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:43.680 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:43.677+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:44.237 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:44.232+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:44.655 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:44.652+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:45.205 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:45.202+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:45.695 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:45.692+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:46.243 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:46.240+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:46.712 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:46.709+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:47.236 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:47.233+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:47.703 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:47.700+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:48.217 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:48.214+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:48.663 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:48.660+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:49.202 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:49.199+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:49.673 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:49.670+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:50.221 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:50.218+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:50.626 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:50.622+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:51.176 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:51.173+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:51.632 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:51.629+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:52.171 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:52.168+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:52.644 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:52.641+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:53.205 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:53.202+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:53.605 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:53.602+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:54.180 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:54.177+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:54.574 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:54.571+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:55.150 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:55.148+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:55.567 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:55.564+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:56.153 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:56.150+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:56.525 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:56.523+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:57.106 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:57.103+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:57.531 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:57.528+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:58.134 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:58.132+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:58.578 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:58.576+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:59.107 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:12:59.104+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:12:59.613 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:12:59.610+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:00.061 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:00.058+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:00.645 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:00.642+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:01.019 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:01.015+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:01.622 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:01.619+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:02.028 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:02.025+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:02.639 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:02.635+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:03.030 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:03.026+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:03.603 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:03.599+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:04.045 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:04.041+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:04.585 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:04.581+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:05.091 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:05.087+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:05.625 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:05.621+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:06.045 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:06.041+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:06.671 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:06.667+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:07.006 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:06.999+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:07.669 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:07.665+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:08.011 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:08.007+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:08.717 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:08.713+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:09.052 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:09.048+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:09.704 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:09.700+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:10.063 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:10.059+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:10.739 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:10.735+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:11.083 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:11.079+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:11.693 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:11.689+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:12.064 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:12.059+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:12.724 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:12.721+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:13.077 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:13.073+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:13.745 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:13.741+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:14.062 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:14.059+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:14.705 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:14.701+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:15.014 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:15.010+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:15.680 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:15.676+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:16.003 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:15.999+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:16.655 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:16.651+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:17.015 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:17.011+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:17.642 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:17.638+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:18.043 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:18.039+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:18.622 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:18.618+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:19.035 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:19.031+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:19.642 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:19.639+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:20.045 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:20.041+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:20.652 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:20.648+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:21.006 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:21.002+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:21.619 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:21.615+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:22.023 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:22.019+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:22.581 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:22.578+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:23.027 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:23.023+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:23.628 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:23.624+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:24.014 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:24.010+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:24.592 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:24.588+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:24.974 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:24.970+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:25.635 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:25.631+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:25.933 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:25.929+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:26.683 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:26.679+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:26.908 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:26.905+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:27.691 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:27.687+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:27.903 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:27.899+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:28.725 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:28.721+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:28.942 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:28.938+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:29.756 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:29.753+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:29.949 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:29.945+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:30.800 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:30.796+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:30.964 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:30.960+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:31.833 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:31.829+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:31.933 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:31.929+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:32.838 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:32.835+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:32.961 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:32.957+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:33.794 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:33.790+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:33.956 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:33.952+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:34.748 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:34.745+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:34.918 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:34.915+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:35.755 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:35.752+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:35.901 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:35.898+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:36.765 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:36.762+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:36.895 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:36.892+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:37.797 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:37.794+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:37.866 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:37.864+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:38.819 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:38.816+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:38.832 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:38.830+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:39.815 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:39.812+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:39.845 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:39.842+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:40.811 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:40.808+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:40.815 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:40.812+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:41.835 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:41.832+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:41.848 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:41.846+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:42.787 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:42.784+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:42.864 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:42.861+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:43.790 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:43.788+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:43.887 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:43.884+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:44.783 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:44.780+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:44.859 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:44.856+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:45.812 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:45.809+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:45.829 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:45.826+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:46.788 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:46.786+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:46.801 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:46.798+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:47.831 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:47.828+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:47.838 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:47.836+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:48.821 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:48.818+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:48.840 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:48.837+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:49.790 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:49.787+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:49.808 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:49.805+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:50.750 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:50.748+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:50.795 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:50.792+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:51.716 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:51.713+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:51.779 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:51.776+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:52.745 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:52.741+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:52.746 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:52.741+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:53.703 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:53.699+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:53.765 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:53.761+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:54.739 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:54.735+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:54.740 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:54.736+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:55.739 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:55.735+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:55.769 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:55.765+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:56.778 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:56.774+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:56.787 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:56.784+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:57.753 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:57.749+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:57.806 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:57.803+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:58.781 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:58.778+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:58.790 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:58.787+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:59.755 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:13:59.751+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:13:59.803 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:13:59.799+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:00.711 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:00.707+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:00.851 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:00.847+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:01.692 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:01.688+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:01.810 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:01.806+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:02.652 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:02.648+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:02.763 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:02.759+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:03.605 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:03.601+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:03.776 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:03.772+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:04.560 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:04.556+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:04.769 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:04.766+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:05.565 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:05.561+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:05.809 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:05.804+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:06.597 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:06.593+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:06.806 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:06.803+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:07.605 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:07.602+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:07.798 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:07.795+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:08.563 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:08.559+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:08.786 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:08.782+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:09.525 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:09.520+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:09.759 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:09.756+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:10.563 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:10.559+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:10.727 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:10.723+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:11.580 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:11.577+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:11.773 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:11.769+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:12.537 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:12.533+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:12.741 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:12.737+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:13.549 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:13.545+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:13.711 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:13.707+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:14.528 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:14.525+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:14.734 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:14.730+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:15.528 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:15.525+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:15.691 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:15.687+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:16.558 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:16.555+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:16.733 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:16.730+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:17.578 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:17.573+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:17.703 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:17.699+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:18.588 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:18.585+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:18.717 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:18.713+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:19.583 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:19.579+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:19.753 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:19.749+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:20.599 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:20.595+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:20.718 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:20.714+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:21.621 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:21.617+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:21.680 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:21.677+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:22.621 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:22.617+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:22.710 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:22.706+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:23.662 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:23.654+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:23.707 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:23.703+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:24.704 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:24.701+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:24.730 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:24.728+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:25.709 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:25.705+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:25.741 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:25.738+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:26.674 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:26.671+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:26.740 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:26.737+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:27.646 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:27.644+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:27.767 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:27.764+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:28.613 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:28.609+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:28.761 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:28.758+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:29.600 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:29.597+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:29.767 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:29.764+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:30.617 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:30.614+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:30.722 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:30.719+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:31.613 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:31.610+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:31.688 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:31.685+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:32.567 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:32.564+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:32.671 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:32.669+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:33.587 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:33.584+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:33.633 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:33.630+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:34.547 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:34.544+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:34.671 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:34.668+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:35.506 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:35.503+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:35.678 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:35.675+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:36.488 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:36.486+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:36.677 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:36.675+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:37.503 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:37.500+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:37.660 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:37.657+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:38.499 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:38.496+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:38.642 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:38.640+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:39.488 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:39.485+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:39.658 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:39.655+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:40.477 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:40.474+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:40.646 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:40.643+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:41.519 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:41.516+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:41.674 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:41.671+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:42.521 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:42.519+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:42.651 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:42.648+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:43.525 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:43.522+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:43.619 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:43.616+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:44.532 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:44.528+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:44.629 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:44.625+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:45.500 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:45.497+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:45.660 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:45.657+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:46.460 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:46.456+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:46.665 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:46.662+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:47.504 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:47.501+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:47.631 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:47.628+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:48.518 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:48.514+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:48.645 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:48.641+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:49.521 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:49.517+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:49.678 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:49.675+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:50.508 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:50.504+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:50.655 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:50.652+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:51.495 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:51.491+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:51.663 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:51.659+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:52.526 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:52.522+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:52.680 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:52.676+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:53.553 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:53.549+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:53.673 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:53.669+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:54.587 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:54.569+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:54.669 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:54.665+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:55.592 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:55.588+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:55.669 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:55.664+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:56.583 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:56.579+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:56.711 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:56.707+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:57.570 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:57.566+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:57.711 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:57.707+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:58.614 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:58.611+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:58.753 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:58.749+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:59.600 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:14:59.597+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:14:59.704 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:14:59.701+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:00.628 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:00.624+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:00.730 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:00.727+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:01.668 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:01.664+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:01.751 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:01.747+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:02.689 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:02.685+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:02.787 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:02.783+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:03.650 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:03.646+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:03.815 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:03.811+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:04.623 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:04.619+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:04.799 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:04.796+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:05.598 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:05.594+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:05.775 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:05.772+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:06.614 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:06.610+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:06.738 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:06.734+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:07.607 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:07.603+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:07.722 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:07.718+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:08.572 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:08.568+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:08.679 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:08.675+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:09.555 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:09.551+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:09.664 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:09.660+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:10.596 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:10.592+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:10.616 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:10.613+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:11.592 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:11.588+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:11.666 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:11.662+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:12.584 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:12.580+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:12.707 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:12.703+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:13.573 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:13.569+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:13.738 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:13.735+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:14.583 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:14.578+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:14.716 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:14.712+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:15.626 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:15.622+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:15.683 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:15.679+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:16.674 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:16.670+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:16.679 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:16.676+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:17.636 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:17.632+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:17.641 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:17.637+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:18.602 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:18.600+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:18.639 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:18.636+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:19.635 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:19.632+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:19.675 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:19.672+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:20.588 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:20.586+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:20.675 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:20.672+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:21.629 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:21.626+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:21.664 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:21.662+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:22.606 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:22.604+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:22.617 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:22.614+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:23.580 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:23.577+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:23.603 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:23.601+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:24.591 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:24.588+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:24.614 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:24.611+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:25.570 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:25.567+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:25.595 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:25.584+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:26.539 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:26.536+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:26.548 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:26.545+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:27.556 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:27.554+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:27.566 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:27.564+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:28.572 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:28.569+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:28.595 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:28.592+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:29.567 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:29.564+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:29.595 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:29.593+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:30.602 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:30.600+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:30.628 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:30.626+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:31.582 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:31.579+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:31.630 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:31.627+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:32.594 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:32.592+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:32.629 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:32.626+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:33.585 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:33.582+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:33.606 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:33.604+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:34.565 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:34.562+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:34.580 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:34.577+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:35.589 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:35.586+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:35.606 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:35.603+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:36.565 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:36.561+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:36.628 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:36.624+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:37.561 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:37.557+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:37.676 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:37.672+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:38.547 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:38.543+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:38.687 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:38.684+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:39.591 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:39.586+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:39.732 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:39.729+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:40.562 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:40.559+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:40.757 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:40.753+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:41.564 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:41.560+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:41.725 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:41.721+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:42.581 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:42.577+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:42.680 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:42.677+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:43.629 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:43.625+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:43.670 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:43.666+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:44.658 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:44.654+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:44.670 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:44.666+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:45.668 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:45.664+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:45.689 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:45.685+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:46.701 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:46.697+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:46.704 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:46.699+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:47.689 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:47.685+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:47.712 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:47.709+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:48.712 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:48.709+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:48.749 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:48.745+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:49.666 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:49.662+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:49.787 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:49.783+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:50.660 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:50.657+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:50.808 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:50.804+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:51.703 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:51.699+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:51.771 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:51.767+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:52.702 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:52.698+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:52.761 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:52.757+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:53.654 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:53.650+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:53.772 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:53.769+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:54.672 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:54.668+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:54.803 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:54.800+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:55.703 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:55.699+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:55.835 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:55.831+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:56.661 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:56.657+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:56.791 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:56.787+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:57.613 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:57.609+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:57.798 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:57.794+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:58.619 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:58.615+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:58.769 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:58.763+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:59.621 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:15:59.618+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:15:59.798 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:15:59.795+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:00.590 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:00.586+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:00.777 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:00.773+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:01.584 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:01.581+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:01.806 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:01.803+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:02.607 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:02.604+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:02.830 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:02.826+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:03.657 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:03.653+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:03.793 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:03.789+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:04.684 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:04.681+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:04.745 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:04.741+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:05.672 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:05.668+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:05.775 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:05.771+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:06.660 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:06.656+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:06.796 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:06.792+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:07.648 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:07.644+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:07.836 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:07.832+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:08.683 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:08.680+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:08.885 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:08.881+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:09.638 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:09.636+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:09.842 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:09.839+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:10.673 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:10.671+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:10.831 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:10.829+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:11.692 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:11.689+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:11.797 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:11.795+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:12.709 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:12.705+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:12.807 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:12.804+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:13.707 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:13.705+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:13.830 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:13.828+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:14.761 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:14.755+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:14.869 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:14.866+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:15.800 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:15.798+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:15.851 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:15.848+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:16.799 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:16.796+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:16.865 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:16.862+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:17.836 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:17.833+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:17.853 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:17.850+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:18.858 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:18.853+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:18.861 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:18.859+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:19.814 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:19.811+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:19.866 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:19.863+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:20.845 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:20.842+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:20.866 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:20.864+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:21.827 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:21.825+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:21.893 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:21.891+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:22.845 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:22.843+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:22.866 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:22.863+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:23.841 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:23.839+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:23.867 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:23.864+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:24.856 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:24.853+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:24.908 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:24.905+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:25.860 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:25.857+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:25.916 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:25.913+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:26.832 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:26.830+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:26.906 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:26.903+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:27.850 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:27.846+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:27.938 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:27.934+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:28.892 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:28.888+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:28.899 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:28.896+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:29.855 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:29.851+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:29.904 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:29.900+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:30.893 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:30.890+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:30.947 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:30.943+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:31.847 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:31.843+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:31.981 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:31.978+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:32.875 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:32.871+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:32.935 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:32.932+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:33.874 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:33.870+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:33.887 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:33.883+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:34.899 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:34.895+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:34.922 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:34.919+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:35.896 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:35.892+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:35.913 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:35.909+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:36.882 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:36.878+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:36.894 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:36.890+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:37.891 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:37.888+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:37.927 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:37.923+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:38.846 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:38.842+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:38.946 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:38.942+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:39.856 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:39.852+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:39.974 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:39.970+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:40.902 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:40.898+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:40.931 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:40.928+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:41.923 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:41.920+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:41.937 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:41.934+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:42.934 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:42.930+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:42.960 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:42.956+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:43.960 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:43.957+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:44.001 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:43.997+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:44.918 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:44.914+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:44.968 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:44.964+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:45.948 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:45.944+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:45.966 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:45.962+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:46.952 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:46.948+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:46.975 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:46.971+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:47.918 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:47.914+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:48.016 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:48.012+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:48.884 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:48.881+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:48.984 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:48.980+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:49.867 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:49.863+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:50.034 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:50.030+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:50.864 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:50.860+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:51.014 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:51.010+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:51.874 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:51.868+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:52.011 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:52.007+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:52.867 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:52.863+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:52.966 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:52.962+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:53.873 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:53.869+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:54.014 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:54.010+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:54.825 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:54.821+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:55.033 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:55.030+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:55.809 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:55.806+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:56.013 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:56.009+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:56.852 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:56.848+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:57.031 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:57.028+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:57.854 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:57.850+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:58.025 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:58.021+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:58.833 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:58.829+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:59.015 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:16:59.011+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:16:59.829 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:16:59.825+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:00.020 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:00.016+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:00.862 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:00.859+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:00.981 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:00.978+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:01.844 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:01.841+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:02.000 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:01.997+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:02.803 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:02.800+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:02.994 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:02.992+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:03.774 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:03.771+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:03.996 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:03.993+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:04.797 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:04.794+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:05.012 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:05.009+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:05.788 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:05.785+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:06.054 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:06.051+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:06.801 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:06.799+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:07.039 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:07.037+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:07.851 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:07.848+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:08.044 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:08.041+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:08.848 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:08.846+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:09.012 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:09.009+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:09.880 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:09.877+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:10.025 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:10.023+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:10.925 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:10.923+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:11.006 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:11.004+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:11.910 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:11.907+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:12.049 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:12.046+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:12.871 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:12.869+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:13.052 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:13.049+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:13.856 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:13.853+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:14.044 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:14.041+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:14.876 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:14.873+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:15.017 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:15.015+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:15.831 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:15.828+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:16.008 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:16.005+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:16.799 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:16.796+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:17.023 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:17.020+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:17.818 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:17.815+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:17.984 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:17.981+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:18.786 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:18.784+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:18.976 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:18.973+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:19.831 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:19.827+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:20.004 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:20.000+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:20.872 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:20.868+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:21.036 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:21.032+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:21.841 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:21.838+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:22.082 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:22.078+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:22.798 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:22.794+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:23.040 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:23.036+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:23.812 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:23.808+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:24.030 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:24.026+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:24.783 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:24.779+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:25.055 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:25.051+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:25.832 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:25.828+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:26.040 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:26.036+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:26.834 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:26.830+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:27.024 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:27.021+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:27.834 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:27.830+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:28.010 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:28.006+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:28.820 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:28.816+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:28.998 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:28.994+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:29.861 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:29.858+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:29.955 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:29.951+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:30.874 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:30.870+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:30.976 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:30.972+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:31.894 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:31.890+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:31.938 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:31.934+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:32.909 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:32.905+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:32.950 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:32.946+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:33.937 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:33.933+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:33.947 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:33.943+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:34.959 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:34.955+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:34.988 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:34.984+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:35.968 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:35.965+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:35.974 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:35.970+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:36.975 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:36.971+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:36.982 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:36.978+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:37.962 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:37.959+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:37.988 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:37.984+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:38.923 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:38.920+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:39.029 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:39.025+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:39.929 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:39.926+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:40.010 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:40.006+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:40.965 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:40.961+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:41.010 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:41.006+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:41.957 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:41.953+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:41.994 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:41.990+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:42.990 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:42.987+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:43.008 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:43.004+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:43.975 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:43.972+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:43.996 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:43.992+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:44.971 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:44.968+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:45.023 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:45.019+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:45.987 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:45.984+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:46.035 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:46.032+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:47.038 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:47.034+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:47.084 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:47.080+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:48.070 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:48.066+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:48.071 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:48.067+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:49.024 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:49.021+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:49.064 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:49.060+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:50.028 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:50.025+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:50.084 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:50.080+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:50.980 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:50.977+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:51.090 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:51.086+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:52.022 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:52.018+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:52.103 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:52.099+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:53.005 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:53.002+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:53.079 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:53.076+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:53.994 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:53.991+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:54.125 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:54.122+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:55.014 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:55.011+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:55.158 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:55.156+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:56.054 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:56.052+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:56.154 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:56.151+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:57.041 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:57.039+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:57.116 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:57.113+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:58.051 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:58.048+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:58.106 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:58.103+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:59.016 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:59.013+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:59.129 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:17:59.127+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:17:59.973 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:17:59.970+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:00.169 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:00.166+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:00.931 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:00.928+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:01.154 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:01.151+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:01.949 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:01.946+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:02.149 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:02.146+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:02.939 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:02.937+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:03.132 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:03.130+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:03.903 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:03.900+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:04.167 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:04.165+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:04.938 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:04.936+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:05.152 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:05.150+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:05.913 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:05.910+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:06.109 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:06.106+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:06.943 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:06.940+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:07.101 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:07.098+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:07.973 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:07.970+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:08.125 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:08.122+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:08.968 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:08.965+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:09.106 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:09.103+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:09.960 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:09.957+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:10.107 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:10.104+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:10.961 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:10.957+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:11.153 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:11.149+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:11.997 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:11.993+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:12.196 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:12.192+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:13.045 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:13.041+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:13.207 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:13.204+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:14.041 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:14.037+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:14.166 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:14.163+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:15.070 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:15.067+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:15.180 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:15.177+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:16.117 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:16.113+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:16.217 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:16.213+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:17.145 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:17.141+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:17.250 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:17.247+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:18.185 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:18.181+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:18.207 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:18.203+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:19.192 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:19.188+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:19.257 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:19.253+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:20.208 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:20.205+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:20.241 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:20.238+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:21.208 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:21.204+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:21.242 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:21.238+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:22.244 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:22.241+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:22.254 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:22.250+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:23.212 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:23.208+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:23.241 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:23.237+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:24.216 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:24.212+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:24.284 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:24.281+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:25.215 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:25.211+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:25.287 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:25.283+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:26.265 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:26.261+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:26.289 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:26.285+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:27.306 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:27.303+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:27.325 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:27.321+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:28.274 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:28.270+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:28.314 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:28.310+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:29.304 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:29.300+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:29.314 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:29.311+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:30.264 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:30.260+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:30.335 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:30.332+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:31.243 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:31.239+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:31.319 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:31.315+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:32.267 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:32.263+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:32.356 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:32.352+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:33.218 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:33.214+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:33.404 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:33.400+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:34.227 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:34.223+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:34.374 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:34.371+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:35.201 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:35.197+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:35.358 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:35.354+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:36.214 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:36.210+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:36.357 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:36.353+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:37.262 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:37.258+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:37.338 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:37.334+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:38.236 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:38.232+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:38.316 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:38.313+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:39.214 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:39.211+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:39.303 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:39.299+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:40.231 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:40.227+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:40.350 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:40.346+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:41.206 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:41.203+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:41.349 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:41.345+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:42.232 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:42.228+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:42.387 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:42.383+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:43.242 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:43.239+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:43.352 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:43.348+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:44.249 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:44.246+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:44.325 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:44.322+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:45.280 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:45.277+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:45.357 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:45.355+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:46.279 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:46.277+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:46.405 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:46.403+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:47.316 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:47.313+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:47.400 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:47.397+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:48.361 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:48.359+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:48.363 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:48.360+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:49.377 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:49.374+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:49.402 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:49.400+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:50.366 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:50.363+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:50.374 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:50.372+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:51.330 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:51.327+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:51.372 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:51.370+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:52.280 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:52.277+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:52.347 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:52.344+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:53.276 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:53.273+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:53.370 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:53.367+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:54.326 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:54.323+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:54.349 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:54.346+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:55.297 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:55.295+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:55.310 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:55.308+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:56.290 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:56.288+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:56.315 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:56.313+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:57.269 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:57.266+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:57.323 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:57.321+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:58.263 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:58.260+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:58.297 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:58.294+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:59.265 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:18:59.262+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:18:59.309 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:18:59.306+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:00.268 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:00.265+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:00.270 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:00.268+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:01.287 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:01.284+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:01.314 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:01.311+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:02.290 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:02.286+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:02.337 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:02.333+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:03.276 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:03.272+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:03.292 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:03.288+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:04.245 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:04.242+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:04.325 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:04.321+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:05.293 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:05.289+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:05.351 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:05.348+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:06.327 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:06.323+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:06.379 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:06.376+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:07.356 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:07.352+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:07.374 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:07.370+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:08.324 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:08.320+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:08.382 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:08.378+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:09.344 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:09.340+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:09.360 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:09.357+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:10.319 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:10.315+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:10.327 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:10.324+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:11.276 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:11.272+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:11.310 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:11.306+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:12.248 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:12.244+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:12.345 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:12.341+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:13.279 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:13.275+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:13.379 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:13.376+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:14.274 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:14.270+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:14.353 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:14.349+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:15.283 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:15.279+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:15.385 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:15.380+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:16.326 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:16.322+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:16.348 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:16.344+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:17.278 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:17.274+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:17.324 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:17.320+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:18.230 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:18.227+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:18.288 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:18.284+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:19.236 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:19.232+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:19.281 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:19.277+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:20.189 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:20.185+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:20.243 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:20.239+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:21.144 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:21.140+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:21.262 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:21.259+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:22.132 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:22.128+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:22.309 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:22.305+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:23.094 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:23.090+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:23.297 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:23.293+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:24.058 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:24.054+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:24.344 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:24.340+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:25.037 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:25.034+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:25.366 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:25.362+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:26.082 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:26.078+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:26.396 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:26.392+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:27.040 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:27.036+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:27.387 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:27.383+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:28.077 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:28.073+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:28.435 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:28.432+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:29.101 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:29.097+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:29.417 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:29.414+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:30.092 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:30.088+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:30.408 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:30.405+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:31.068 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:31.064+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:31.444 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:31.441+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:32.031 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:32.027+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:32.482 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:32.478+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:33.040 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:33.037+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:33.458 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:33.454+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:34.083 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:34.080+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:34.506 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:34.502+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:35.113 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:35.110+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:35.547 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:35.543+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:36.154 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:36.152+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:36.531 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:36.529+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:37.185 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:37.182+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:37.548 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:37.545+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:38.137 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:38.135+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:38.575 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:38.573+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:39.126 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:39.123+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:39.616 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:39.613+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:40.124 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:40.121+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:40.637 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:40.635+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:41.108 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:41.105+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:41.663 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:41.660+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:42.098 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:42.095+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:42.649 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:42.646+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:43.094 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:43.091+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:43.695 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:43.691+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:44.066 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:44.063+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:44.704 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:44.701+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:45.041 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:45.039+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:45.737 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:45.734+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:46.015 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:46.012+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:46.751 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:46.748+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:46.981 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:46.978+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:47.789 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:47.786+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:48.001 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:47.998+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:48.780 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:48.777+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:48.985 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:48.982+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:49.757 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:49.755+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:50.022 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:50.019+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:50.762 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:50.760+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:50.999 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:50.997+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:51.742 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:51.740+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:52.004 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:52.001+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:52.776 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:52.773+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:53.017 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:53.015+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:53.796 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:53.792+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:53.978 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:53.974+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:54.795 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:54.791+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:54.988 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:54.984+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:55.755 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:55.752+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:55.980 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:55.976+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:56.775 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:56.771+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:57.005 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:57.002+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:57.763 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:57.760+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:58.007 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:58.003+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:58.749 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:58.745+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:58.959 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:58.955+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:59.729 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:19:59.726+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:19:59.936 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:19:59.932+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:00.758 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:00.754+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:00.928 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:00.924+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:01.753 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:01.750+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:01.952 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:01.949+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:02.740 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:02.736+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:02.959 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:02.956+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:03.775 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:03.771+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:03.934 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:03.930+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:04.758 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:04.754+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:04.924 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:04.921+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:05.744 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:05.740+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:05.952 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:05.948+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:06.735 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:06.731+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:07.002 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:06.998+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:07.772 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:07.768+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:08.036 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:08.032+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:08.764 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:08.760+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:08.999 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:08.996+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:09.727 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:09.724+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:10.003 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:10.000+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:10.743 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:10.739+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:11.003 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:11.000+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:11.790 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:11.786+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:11.961 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:11.958+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:12.759 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:12.755+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:12.974 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:12.968+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:13.751 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:13.748+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:13.932 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:13.928+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:14.780 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:14.776+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:14.966 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:14.962+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:15.750 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:15.746+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:15.966 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:15.962+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:16.711 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:16.708+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:16.925 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:16.922+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:17.664 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:17.660+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:17.930 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:17.927+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:18.665 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:18.661+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:18.913 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:18.909+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:19.623 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:19.619+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:19.949 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:19.946+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:20.664 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:20.660+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:20.936 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:20.932+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:21.619 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:21.615+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:21.942 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:21.939+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:22.668 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:22.665+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:22.982 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:22.978+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:23.716 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:23.712+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:23.965 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:23.962+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:24.674 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:24.671+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:24.960 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:24.956+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:25.634 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:25.630+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:26.000 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:25.997+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:26.627 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:26.624+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:27.017 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:27.013+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:27.665 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:27.662+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:27.990 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:27.987+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:28.659 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:28.656+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:28.946 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:28.944+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:29.616 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:29.613+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:29.991 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:29.988+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:30.569 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:30.566+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:30.986 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:30.984+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:31.598 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:31.596+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:32.004 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:32.001+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:32.637 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:32.633+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:33.003 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:33.000+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:33.625 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:33.623+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:34.023 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:34.020+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:34.620 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:34.618+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:35.057 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:35.054+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:35.594 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:35.592+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:36.095 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:36.092+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:36.558 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:36.555+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:37.064 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:37.061+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:37.512 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:37.509+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:38.021 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:38.018+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:38.536 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:38.533+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:39.068 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:39.065+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:39.540 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:39.537+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:40.087 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:40.085+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:40.556 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:40.554+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:41.070 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:41.068+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:41.535 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:41.533+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:42.093 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:42.090+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:42.505 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:42.503+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:43.119 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:43.116+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:43.459 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:43.456+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:44.123 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:44.120+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:44.424 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:44.422+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:45.076 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:45.073+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:45.430 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:45.426+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:46.066 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:46.063+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:46.404 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:46.400+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:47.035 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:47.031+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:47.355 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:47.351+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:48.046 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:48.042+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:48.335 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:48.331+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:49.093 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:49.089+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:49.344 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:49.340+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:50.063 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:50.059+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:50.390 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:50.386+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:51.051 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:51.047+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:51.429 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:51.425+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:52.086 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:52.083+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:52.476 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:52.473+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:53.063 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:53.059+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:53.506 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:53.502+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:54.038 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:54.034+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:54.535 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:54.531+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:55.067 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:55.064+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:55.528 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:55.524+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:56.088 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:56.084+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:56.519 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:56.515+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:57.085 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:57.081+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:57.558 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:57.554+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:58.065 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:58.061+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:58.559 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:58.555+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:59.112 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:20:59.108+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:20:59.587 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:20:59.583+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:00.107 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:00.103+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:00.555 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:00.551+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:01.127 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:01.124+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:01.591 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:01.587+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:02.114 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:02.110+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:02.608 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:02.604+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:03.160 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:03.157+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:03.569 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:03.565+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:04.115 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:04.111+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:04.587 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:04.583+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:05.078 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:05.074+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:05.570 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:05.566+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:06.075 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:06.072+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:06.605 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:06.601+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:07.068 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:07.064+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:07.655 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:07.650+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:08.058 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:08.054+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:08.640 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:08.636+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:09.014 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:09.010+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:09.666 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:09.663+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:09.965 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:09.962+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:10.703 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:10.700+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:10.994 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:10.990+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:11.654 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:11.651+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:11.992 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:11.989+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:12.683 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:12.679+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:13.006 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:13.002+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:13.672 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:13.669+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:14.006 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:14.003+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:14.670 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:14.666+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:15.022 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:15.018+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:15.675 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:15.672+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:15.998 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:15.995+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:16.633 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:16.629+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:16.963 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:16.959+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:17.663 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:17.659+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:17.951 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:17.948+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:18.692 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:18.689+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:18.944 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:18.942+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:19.671 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:19.668+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:19.922 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:19.919+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:20.694 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:20.691+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:20.882 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:20.880+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:21.729 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:21.727+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:21.851 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:21.849+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:22.775 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:22.773+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:22.852 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:22.849+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:23.733 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:23.730+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:23.827 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:23.824+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:24.735 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:24.732+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:24.855 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:24.853+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:25.722 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:25.719+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:25.867 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:25.864+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:26.709 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:26.706+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:26.830 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:26.827+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:27.703 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:27.700+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:27.850 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:27.847+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:28.701 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:28.698+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:28.876 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:28.873+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:29.692 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:29.689+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:29.842 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:29.839+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:30.698 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:30.695+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:30.837 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:30.835+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:31.721 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:31.719+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:31.830 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:31.827+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:32.675 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:32.672+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:32.841 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:32.839+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:33.722 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:33.719+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:33.808 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:33.805+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:34.714 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:34.711+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:34.832 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:34.829+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:35.722 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:35.719+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:35.843 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:35.840+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:36.694 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:36.690+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:36.814 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:36.810+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:37.651 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:37.647+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:37.818 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:37.814+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:38.623 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:38.619+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:38.791 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:38.787+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:39.617 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:39.613+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:39.774 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:39.771+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:40.664 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:40.660+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:40.760 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:40.756+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:41.675 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:41.671+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:41.779 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:41.775+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:42.692 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:42.689+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:42.785 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:42.781+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:43.711 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:43.707+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:43.822 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:43.819+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:44.704 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:44.700+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:44.779 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:44.776+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:45.682 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:45.678+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:45.760 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:45.756+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:46.699 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:46.695+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:46.807 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:46.803+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:47.745 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:47.742+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:47.794 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:47.790+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:48.788 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:48.784+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:48.834 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:48.831+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:49.806 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:49.802+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:49.828 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:49.824+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:50.820 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:50.816+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:50.839 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:50.836+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:51.854 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:51.850+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:51.874 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:51.871+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:52.819 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:52.815+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:52.892 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:52.888+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:53.858 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:53.854+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:53.936 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:53.932+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:54.848 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:54.844+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:54.900 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:54.896+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:55.857 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:55.854+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:55.882 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:55.878+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:56.811 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:56.808+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:56.914 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:56.910+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:57.815 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:57.812+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:57.906 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:57.902+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:58.805 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:58.802+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:58.922 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:58.918+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:59.790 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:21:59.786+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:21:59.894 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:21:59.889+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:00.772 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:00.769+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:00.870 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:00.867+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:01.819 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:01.816+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:01.872 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:01.869+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:02.864 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:02.860+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:02.885 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:02.882+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:03.889 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:03.886+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:03.914 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:03.910+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:04.923 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:04.919+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:04.925 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:04.922+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:05.935 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:05.931+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:05.956 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:05.953+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:06.958 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:06.955+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:06.975 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:06.972+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:07.946 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:07.942+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:07.969 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:07.965+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:08.991 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:08.987+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:08.992 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:08.987+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:09.965 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:09.962+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:09.987 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:09.984+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:11.006 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:11.003+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:11.007 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:11.004+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:11.991 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:11.988+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:11.998 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:11.996+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:12.956 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:12.953+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:12.989 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:12.986+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:13.980 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:13.977+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:13.989 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:13.986+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:14.985 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:14.982+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:15.026 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:15.023+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:15.948 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:15.945+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:15.979 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:15.976+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:16.975 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:16.972+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:17.029 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:17.026+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:17.958 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:17.956+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:18.066 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:18.063+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:18.978 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:18.975+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:19.047 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:19.044+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:19.967 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:19.964+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:20.040 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:20.038+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:20.941 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:20.938+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:21.067 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:21.065+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:21.935 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:21.932+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:22.050 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:22.047+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:22.949 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:22.946+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:23.065 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:23.062+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:23.955 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:23.953+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:24.111 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:24.109+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:24.946 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:24.943+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:25.107 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:25.104+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:25.928 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:25.925+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:26.149 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:26.146+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:26.911 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:26.908+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:27.197 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:27.194+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:27.885 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:27.881+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:28.174 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:28.170+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:28.929 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:28.925+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:29.195 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:29.191+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:29.940 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:29.936+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:30.192 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:30.188+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:30.900 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:30.896+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:31.226 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:31.222+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:31.861 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:31.857+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:32.266 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:32.263+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:32.866 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:32.862+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:33.273 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:33.269+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:33.906 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:33.903+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:34.300 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:34.296+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:34.877 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:34.873+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:35.290 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:35.286+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:35.907 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:35.903+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:36.295 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:36.291+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:36.895 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:36.891+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:37.273 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:37.269+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:37.855 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:37.851+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:38.315 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:38.312+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:38.904 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:38.901+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:39.305 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:39.301+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:39.938 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:39.934+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:40.336 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:40.333+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:40.957 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:40.953+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:41.300 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:41.297+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:42.001 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:41.997+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:42.348 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:42.345+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:42.966 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:42.962+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:43.316 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:43.312+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:43.948 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:43.944+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:44.351 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:44.348+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:44.917 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:44.914+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:45.398 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:45.394+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:45.924 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:45.921+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:46.390 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:46.386+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:46.903 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:46.900+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:47.375 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:47.371+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:47.912 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:47.908+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:48.382 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:48.378+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:48.900 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:48.896+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:49.421 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:49.418+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:49.895 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:49.892+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:50.385 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:50.381+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:50.865 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:50.861+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:51.350 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:51.347+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:51.871 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:51.867+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:52.366 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:52.362+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:52.900 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:52.896+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:53.318 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:53.314+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:53.899 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:53.895+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:54.362 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:54.358+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:54.931 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:54.927+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:55.397 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:55.393+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:55.947 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:55.943+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:56.416 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:56.412+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:56.929 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:56.925+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:57.450 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:57.447+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:57.943 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:57.939+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:58.408 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:58.405+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:58.945 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:58.941+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:59.373 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:22:59.369+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:22:59.936 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:22:59.932+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:00.342 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:00.339+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:00.948 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:00.944+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:01.304 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:01.301+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:01.970 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:01.967+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:02.280 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:02.277+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:02.962 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:02.959+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:03.298 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:03.296+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:03.999 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:03.997+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:04.289 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:04.287+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:05.002 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:05.000+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:05.295 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:05.293+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:06.010 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:06.007+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:06.311 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:06.309+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:07.011 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:07.008+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:07.357 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:07.355+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:07.989 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:07.987+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:08.358 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:08.355+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:08.945 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:08.943+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:09.362 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:09.360+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:09.969 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:09.967+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:10.373 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:10.370+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:10.935 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:10.932+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:11.369 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:11.367+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:11.957 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:11.954+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:12.389 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:12.387+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:12.997 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:12.995+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:13.425 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:13.422+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:14.020 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:14.017+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:14.416 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:14.413+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:15.018 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:15.015+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:15.376 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:15.373+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:16.056 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:16.053+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:16.380 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:16.377+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:17.040 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:17.037+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:17.426 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:17.423+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:17.996 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:17.993+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:18.382 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:18.379+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:19.011 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:19.007+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:19.386 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:19.382+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:20.020 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:20.016+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:20.362 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:20.358+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:21.052 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:21.048+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:21.406 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:21.402+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:22.045 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:22.041+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:22.382 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:22.378+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:23.086 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:23.082+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:23.429 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:23.426+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:24.100 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:24.096+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:24.451 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:24.447+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:25.108 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:25.104+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:25.412 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:25.408+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:26.152 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:26.148+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:26.413 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:26.409+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:27.185 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:27.182+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:27.431 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:27.427+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:28.213 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:28.210+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:28.386 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:28.382+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:29.252 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:29.248+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:29.346 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:29.342+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:30.255 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:30.251+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:30.380 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:30.376+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:31.207 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:31.204+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:31.388 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:31.384+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:32.203 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:32.199+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:32.362 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:32.358+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:33.155 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:33.151+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:33.374 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:33.370+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:34.141 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:34.137+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:34.401 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:34.397+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:35.099 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:35.096+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:35.355 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:35.351+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:36.094 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:36.090+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:36.395 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:36.392+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:37.132 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:37.128+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:37.368 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:37.364+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:38.094 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:38.090+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:38.335 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:38.332+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:39.089 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:39.085+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:39.345 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:39.342+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:40.121 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:40.117+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:40.328 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:40.324+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:41.147 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:41.144+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:41.280 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:41.276+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:42.116 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:42.112+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:42.232 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:42.228+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:43.069 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:43.066+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:43.209 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:43.205+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:44.086 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:44.083+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:44.171 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:44.167+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:45.078 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:45.075+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:45.198 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:45.194+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:46.059 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:46.055+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:46.158 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:46.154+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:47.051 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:47.048+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:47.151 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:47.147+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:48.038 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:48.035+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:48.181 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:48.177+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:49.033 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:49.029+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:49.229 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:49.225+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:49.997 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:49.994+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:50.274 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:50.270+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:51.033 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:51.029+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:51.228 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:51.224+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:52.034 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:52.030+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:52.180 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:52.176+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:53.068 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:53.065+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:53.222 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:53.219+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:54.111 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:54.108+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:54.268 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:54.265+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:55.114 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:55.111+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:55.280 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:55.277+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:56.142 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:56.140+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:56.317 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:56.314+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:57.146 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:57.143+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:57.346 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:57.343+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:58.117 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:58.114+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:58.314 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:58.311+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:59.068 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:23:59.065+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:23:59.279 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:23:59.276+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:00.039 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:00.036+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:00.302 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:00.300+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:01.057 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:01.054+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:01.286 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:01.283+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:02.106 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:02.103+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:02.238 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:02.235+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:03.096 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:03.093+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:03.211 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:03.208+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:04.104 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:04.101+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:04.222 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:04.219+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:05.066 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:05.063+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:05.199 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:05.197+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:06.115 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:06.112+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:06.250 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:06.247+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:07.156 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:07.153+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:07.228 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:07.225+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:08.172 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:08.169+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:08.235 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:08.232+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:09.133 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:09.129+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:09.207 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:09.205+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:10.110 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:10.107+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:10.178 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:10.175+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:11.156 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:11.137+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:11.169 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:11.165+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:12.134 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:12.130+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:12.211 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:12.207+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:13.182 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:13.178+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:13.245 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:13.241+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:14.178 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:14.174+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:14.260 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:14.256+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:15.222 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:15.218+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:15.298 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:15.294+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:16.254 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:16.250+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:16.295 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:16.291+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:17.276 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:17.272+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:17.301 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:17.297+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:18.232 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:18.228+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:18.258 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:18.254+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:19.203 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:19.199+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:19.225 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:19.221+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:20.180 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:20.176+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:20.247 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:20.244+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:21.203 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:21.199+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:21.214 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:21.211+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:22.174 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:22.170+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:22.197 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:22.194+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:23.131 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:23.127+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:23.222 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:23.219+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:24.112 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:24.108+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:24.214 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:24.210+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:25.084 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:25.081+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:25.215 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:25.212+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:26.121 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:26.118+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:26.251 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:26.247+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:27.075 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:27.072+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:27.230 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:27.226+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:28.069 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:28.065+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:28.217 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:28.214+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:29.029 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:29.025+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:29.187 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:29.184+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:30.058 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:30.054+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:30.179 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:30.176+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:31.087 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:31.084+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:31.151 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:31.148+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:32.088 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:32.084+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:32.199 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:32.195+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:33.062 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:33.058+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:33.219 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:33.215+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:34.108 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:34.104+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:34.178 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:34.174+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:35.114 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:35.110+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:35.179 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:35.175+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:36.104 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:36.101+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:36.194 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:36.191+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:37.087 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:37.083+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:37.171 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:37.168+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:38.104 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:38.100+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:38.145 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:38.142+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:39.060 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:39.056+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:39.157 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:39.152+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:40.108 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:40.104+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:40.144 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:40.140+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:41.074 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:41.070+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:41.114 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:41.110+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:42.095 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:42.092+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:42.156 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:42.153+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:43.053 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:43.049+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:43.161 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:43.157+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:44.091 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:44.089+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:44.161 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:44.158+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:45.119 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:45.116+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:45.134 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:45.131+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:46.102 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:46.099+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:46.150 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:46.147+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:47.149 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:47.146+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:47.176 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:47.173+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:48.138 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:48.135+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:48.180 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:48.177+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:49.160 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:49.157+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:49.173 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:49.171+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:50.114 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:50.110+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:50.208 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:50.205+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:51.078 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:51.075+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:51.258 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:51.255+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:52.046 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:52.043+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:52.305 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:52.302+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:53.064 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:53.061+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:53.296 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:53.293+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:54.076 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:54.073+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:54.324 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:54.321+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:55.110 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:55.107+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:55.325 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:55.322+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:56.141 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:56.139+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:56.318 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:56.315+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:57.180 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:57.177+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:57.327 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:57.324+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:58.213 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:58.211+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:58.301 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:58.299+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:59.188 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:24:59.185+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:24:59.254 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:24:59.251+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:00.185 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:00.183+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:00.277 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:00.275+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:01.224 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:01.221+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:01.278 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:01.275+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:02.265 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:02.261+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:02.293 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:02.289+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:03.272 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:03.268+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:03.282 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:03.279+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:04.275 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:04.272+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:04.305 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:04.302+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:05.235 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:05.231+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:05.290 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:05.286+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:06.237 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:06.234+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:06.311 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:06.307+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:07.193 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:07.189+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:07.314 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:07.310+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:08.209 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:08.206+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:08.300 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:08.297+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:09.175 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:09.171+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:09.297 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:09.294+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:10.211 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:10.208+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:10.288 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:10.284+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:11.201 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:11.197+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:11.269 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:11.265+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:12.246 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:12.242+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:12.256 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:12.252+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:13.248 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:13.244+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:13.280 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:13.277+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:14.259 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:14.256+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:14.269 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:14.266+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:15.216 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:15.212+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:15.230 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:15.226+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:16.200 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:16.196+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:16.230 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:16.223+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:17.179 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:17.175+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:17.230 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:17.226+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:18.162 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:18.158+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:18.263 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:18.259+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:19.114 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:19.111+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:19.224 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:19.220+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:20.101 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:20.097+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:20.227 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:20.224+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:21.133 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:21.129+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:21.278 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:21.274+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:22.153 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:22.149+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:22.283 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:22.279+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:23.173 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:23.169+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:23.257 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:23.254+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:24.128 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:24.124+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:24.210 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:24.206+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:25.168 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:25.165+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:25.260 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:25.256+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:26.177 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:26.173+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:26.231 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:26.228+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:27.189 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:27.186+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:27.230 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:27.226+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:28.215 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:28.211+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:28.253 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:28.249+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:29.223 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:29.219+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:29.261 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:29.257+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:30.231 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:30.228+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:30.254 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:30.250+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:31.238 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:31.234+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:31.243 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:31.240+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:32.225 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:32.221+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:32.236 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:32.233+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:33.209 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:33.205+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:33.212 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:33.209+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:34.214 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:34.210+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:34.225 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:34.222+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:35.181 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:35.178+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:35.254 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:35.250+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:36.198 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:36.195+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:36.252 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:36.249+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:37.184 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:37.181+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:37.231 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:37.228+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:38.144 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:38.141+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:38.200 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:38.197+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:39.181 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:39.178+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:39.249 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:39.246+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:40.189 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:40.186+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:40.295 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:40.293+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:41.221 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:41.218+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:41.329 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:41.326+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:42.185 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:42.182+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:42.368 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:42.366+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:43.206 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:43.203+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:43.325 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:43.322+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:44.158 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:44.156+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:44.339 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:44.336+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:45.123 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:45.120+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:45.353 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:45.350+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:46.120 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:46.118+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:46.379 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:46.376+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:47.116 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:47.113+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:47.374 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:47.372+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:48.143 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:48.140+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:48.382 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:48.379+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:49.189 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:49.187+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:49.426 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:49.423+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:50.232 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:50.230+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:50.399 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:50.396+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:51.223 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:51.220+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:51.386 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:51.384+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:52.206 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:52.203+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:52.432 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:52.429+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:53.235 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:53.232+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:53.391 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:53.388+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:54.278 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:54.275+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:54.400 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:54.396+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:55.308 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:55.305+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:55.380 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:55.377+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:56.284 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:56.281+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:56.333 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:56.329+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:57.327 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:57.323+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:57.373 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:57.369+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:58.298 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:58.295+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:58.369 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:58.365+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:59.327 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:25:59.323+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:25:59.338 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:25:59.334+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:00.311 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:00.307+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:00.326 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:00.322+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:01.265 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:01.261+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:01.292 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:01.289+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:02.251 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:02.247+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:02.305 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:02.301+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:03.222 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:03.218+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:03.304 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:03.300+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:04.259 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:04.255+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:04.302 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:04.298+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:05.269 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:05.265+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:05.285 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:05.281+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:06.240 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:06.237+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:06.313 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:06.309+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:07.224 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:07.220+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:07.270 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:07.267+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:08.269 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:08.265+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:08.313 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:08.309+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:09.289 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:09.285+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:09.315 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:09.312+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:10.318 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:10.314+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:10.342 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:10.338+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:11.304 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:11.301+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:11.363 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:11.359+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:12.318 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:12.314+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:12.353 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:12.349+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:13.270 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:13.267+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:13.363 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:13.359+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:14.253 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:14.249+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:14.362 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:14.358+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:15.229 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:15.225+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:15.361 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:15.357+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:16.184 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:16.181+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:16.367 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:16.364+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:17.212 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:17.209+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:17.409 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:17.405+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:18.185 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:18.181+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:18.424 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:18.420+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:19.146 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:19.143+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:19.459 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:19.455+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:20.105 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:20.101+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:20.427 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:20.423+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:21.137 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:21.134+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:21.396 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:21.392+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:22.088 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:22.084+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:22.414 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:22.410+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:23.066 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:23.062+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:23.451 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:23.447+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:24.071 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:24.067+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:24.451 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:24.447+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:25.118 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:25.114+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:25.424 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:25.420+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:26.071 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:26.068+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:26.390 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:26.386+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:27.087 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:27.083+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:27.348 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:27.344+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:28.040 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:28.037+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:28.383 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:28.380+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:29.000 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:28.998+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:29.385 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:29.383+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:30.024 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:30.021+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:30.405 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:30.403+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:31.053 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:31.050+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:31.404 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:31.401+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:32.025 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:32.022+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:32.404 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:32.401+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:32.984 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:32.981+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:33.372 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:33.369+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:33.967 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:33.965+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:34.407 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:34.404+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:35.005 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:35.002+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:35.425 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:35.423+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:36.012 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:36.009+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:36.462 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:36.459+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:37.066 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:36.995+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:37.422 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:37.419+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:37.965 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:37.962+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:38.386 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:38.383+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:38.974 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:38.971+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:39.410 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:39.407+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:39.933 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:39.930+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:40.367 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:40.365+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:40.962 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:40.960+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:41.346 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:41.343+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:41.969 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:41.966+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:42.320 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:42.317+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:42.990 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:42.988+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:43.340 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:43.337+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:43.974 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:43.971+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:44.297 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:44.295+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:44.936 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:44.933+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:45.282 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:45.279+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:45.922 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:45.919+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:46.281 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:46.277+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:46.945 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:46.941+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:47.234 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:47.230+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:47.915 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:47.911+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:48.281 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:48.277+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:48.937 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:48.934+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:49.239 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:49.236+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:49.897 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:49.893+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:50.285 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:50.282+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:50.898 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:50.895+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:51.299 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:51.295+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:51.911 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:51.908+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:52.260 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:52.256+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:52.866 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:52.862+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:53.304 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:53.300+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:53.871 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:53.868+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:54.287 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:54.284+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:54.905 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:54.901+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:55.337 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:55.333+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:55.938 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:55.934+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:56.349 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:56.345+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:56.891 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:56.887+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:57.358 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:57.354+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:57.937 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:57.934+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:58.351 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:58.347+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:58.942 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:58.938+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:59.313 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:26:59.310+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:26:59.903 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:26:59.900+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:00.357 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:00.354+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:00.895 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:00.891+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:01.389 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:01.386+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:01.873 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:01.870+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:02.360 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:02.356+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:02.846 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:02.842+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:03.334 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:03.330+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:03.850 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:03.846+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:04.325 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:04.322+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:04.835 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:04.832+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:05.340 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:05.336+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:05.834 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:05.830+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:06.367 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:06.364+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:06.806 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:06.802+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:07.412 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:07.409+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:07.812 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:07.809+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:08.464 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:08.408+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:08.833 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:08.830+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:09.383 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:09.379+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:09.811 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:09.807+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:10.415 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:10.412+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:10.813 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:10.809+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:11.389 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:11.386+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:11.835 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:11.831+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:12.347 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:12.344+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:12.820 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:12.816+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:13.388 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:13.384+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:13.835 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:13.831+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:14.416 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:14.412+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:14.844 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:14.840+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:15.423 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:15.419+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:15.818 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:15.815+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:16.382 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:16.379+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:16.793 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:16.789+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:17.342 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:17.338+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:17.779 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:17.776+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:18.317 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:18.314+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:18.748 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:18.745+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:19.295 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:19.292+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:19.701 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:19.699+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:20.270 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:20.267+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:20.695 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:20.692+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:21.285 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:21.283+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:21.647 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:21.644+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:22.333 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:22.330+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:22.697 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:22.695+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:23.298 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:23.296+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:23.746 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:23.744+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:24.265 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:24.262+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:24.795 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:24.792+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:25.310 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:25.308+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:25.766 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:25.763+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:26.285 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:26.282+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:26.815 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:26.813+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:27.312 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:27.309+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:27.789 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:27.786+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:28.302 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:28.299+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:28.818 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:28.815+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:29.261 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:29.258+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:29.821 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:29.818+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:30.239 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:30.236+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:30.794 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:30.791+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:31.208 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:31.205+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:31.831 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:31.828+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:32.244 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:32.241+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:32.814 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:32.812+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:33.266 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:33.263+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:33.772 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:33.769+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:34.272 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:34.269+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:34.782 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:34.779+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:35.246 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:35.244+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:35.816 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:35.813+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:36.286 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:36.284+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:36.849 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:36.847+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:37.249 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:37.246+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:37.886 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:37.882+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:38.248 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:38.244+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:38.931 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:38.927+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:39.237 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:39.234+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:39.944 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:39.941+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:40.285 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:40.281+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:40.945 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:40.941+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:41.236 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:41.233+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:41.906 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:41.902+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:42.204 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:42.201+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:42.881 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:42.877+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:43.197 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:43.193+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:43.899 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:43.895+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:44.152 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:44.148+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:44.949 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:44.945+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:45.177 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:45.173+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:45.981 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:45.978+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:46.180 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:46.177+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:46.986 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:46.982+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:47.174 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:47.171+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:48.013 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:48.009+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:48.168 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:48.165+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:49.025 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:49.021+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:49.200 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:49.196+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:50.046 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:50.042+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:50.181 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:50.177+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:51.044 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:51.041+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:51.149 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:51.145+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:52.046 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:52.043+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:52.128 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:52.124+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:53.002 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:52.998+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:53.163 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:53.159+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:54.008 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:54.005+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:54.180 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:54.176+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:55.025 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:55.021+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:55.218 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:55.214+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:55.984 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:55.981+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:56.210 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:56.206+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:56.967 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:56.963+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:57.238 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:57.234+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:57.930 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:57.926+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:58.273 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:58.270+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:58.942 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:58.939+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:59.224 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:27:59.221+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:27:59.957 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:27:59.953+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:00.248 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:00.245+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:00.950 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:00.947+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:01.274 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:01.271+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:01.970 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:01.966+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:02.276 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:02.272+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:02.975 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:02.972+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:03.240 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:03.236+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:03.963 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:03.959+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:04.271 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:04.267+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:04.970 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:04.967+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:05.314 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:05.310+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:05.950 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:05.947+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:06.289 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:06.285+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:06.979 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:06.975+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:07.307 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:07.303+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:07.935 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:07.931+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:08.289 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:08.285+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:08.982 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:08.979+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:09.300 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:09.296+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:09.993 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:09.990+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:10.265 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:10.261+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:11.041 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:11.038+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:11.223 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:11.220+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:12.025 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:12.021+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:12.245 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:12.243+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:13.026 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:13.024+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:13.278 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:13.275+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:14.069 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:14.066+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:14.311 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:14.308+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:15.051 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:15.048+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:15.277 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:15.274+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:16.012 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:16.009+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:16.253 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:16.251+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:16.998 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:16.996+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:17.248 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:17.245+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:18.019 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:18.016+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:18.240 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:18.238+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:18.986 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:18.984+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:19.288 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:19.285+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:20.035 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:20.032+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:20.271 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:20.269+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:21.007 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:21.004+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:21.258 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:21.255+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:22.036 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:22.034+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:22.298 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:22.296+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:23.027 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:23.025+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:23.286 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:23.283+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:24.029 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:24.026+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:24.247 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:24.244+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:25.021 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:25.019+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:25.244 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:25.242+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:26.012 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:26.009+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:26.288 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:26.286+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:26.984 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:26.981+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:27.265 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:27.262+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:27.943 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:27.940+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:28.299 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:28.296+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:28.960 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:28.957+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:29.336 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:29.333+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:30.008 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:30.005+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:30.383 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:30.380+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:31.030 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:31.026+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:31.427 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:31.424+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:32.059 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:32.056+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:32.437 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:32.434+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:33.088 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:33.084+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:33.477 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:33.473+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:34.068 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:34.065+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:34.477 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:34.473+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:35.100 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:35.096+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:35.471 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:35.467+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:36.058 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:36.054+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:36.502 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:36.499+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:37.023 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:37.019+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:37.484 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:37.480+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:38.036 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:38.033+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:38.483 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:38.479+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:39.045 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:39.041+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:39.529 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:39.525+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:39.997 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:39.994+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:40.562 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:40.558+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:40.988 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:40.984+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:41.601 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:41.598+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:42.031 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:42.028+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:42.619 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:42.616+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:43.035 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:43.031+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:43.605 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:43.602+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:43.990 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:43.987+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:44.655 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:44.651+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:44.988 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:44.984+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:45.614 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:45.611+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:45.962 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:45.958+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:46.641 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:46.638+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:46.948 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:46.944+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:47.615 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:47.612+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:47.926 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:47.922+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:48.579 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:48.575+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:48.975 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:48.972+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:49.601 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:49.597+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:49.940 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:49.937+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:50.626 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:50.622+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:50.896 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:50.892+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:51.627 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:51.623+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:51.859 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:51.856+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:52.630 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:52.626+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:52.838 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:52.835+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:53.622 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:53.619+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:53.798 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:53.794+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:54.644 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:54.640+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:54.789 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:54.786+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:55.647 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:55.644+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:55.802 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:55.799+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:56.661 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:56.658+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:56.789 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:56.786+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:57.614 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:57.610+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:57.774 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:57.771+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:58.609 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:58.606+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:58.751 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:58.747+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:59.563 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:28:59.560+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:28:59.790 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:28:59.787+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:00.566 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:00.562+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:00.786 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:00.782+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:01.607 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:01.604+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:01.769 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:01.766+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:02.654 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:02.650+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:02.742 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:02.739+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:03.704 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:03.700+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:03.750 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:03.747+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:04.724 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:04.721+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:04.725 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:04.722+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:05.686 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:05.683+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:05.708 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:05.706+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:06.669 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:06.666+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:06.709 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:06.706+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:07.661 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:07.658+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:07.741 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:07.738+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:08.698 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:08.696+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:08.767 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:08.764+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:09.680 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:09.677+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:09.760 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:09.758+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:10.717 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:10.714+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:10.766 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:10.763+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:11.713 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:11.710+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:11.815 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:11.813+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:12.705 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:12.702+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:12.820 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:12.817+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:13.660 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:13.658+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:13.779 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:13.776+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:14.686 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:14.683+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:14.809 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:14.806+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:15.705 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:15.703+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:15.806 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:15.804+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:16.708 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:16.705+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:16.824 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:16.822+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:17.680 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:17.678+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:17.804 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:17.802+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:18.703 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:18.700+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:18.833 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:18.831+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:19.657 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:19.655+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:19.848 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:19.846+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:20.646 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:20.644+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:20.877 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:20.874+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:21.610 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:21.607+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:21.923 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:21.921+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:22.634 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:22.632+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:22.898 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:22.896+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:23.655 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:23.651+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:23.864 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:23.861+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:24.613 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:24.610+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:24.895 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:24.891+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:25.594 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:25.591+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:25.893 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:25.889+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:26.575 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:26.571+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:26.899 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:26.896+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:27.580 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:27.576+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:27.857 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:27.853+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:28.618 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:28.615+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:28.898 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:28.895+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:29.666 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:29.663+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:29.894 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:29.891+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:30.662 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:30.659+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:30.913 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:30.910+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:31.660 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:31.657+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:31.881 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:31.877+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:32.677 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:32.673+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:32.867 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:32.863+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:33.711 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:33.707+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:33.902 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:33.898+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:34.696 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:34.693+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:34.940 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:34.936+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:35.672 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:35.669+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:35.908 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:35.905+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:36.696 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:36.693+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:36.874 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:36.871+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:37.674 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:37.670+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:37.888 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:37.885+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:38.712 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:38.708+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:38.936 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:38.933+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:39.683 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:39.680+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:39.975 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:39.971+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:40.641 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:40.637+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:40.972 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:40.969+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:41.630 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:41.626+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:42.015 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:42.012+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:42.592 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:42.589+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:43.022 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:43.019+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:43.596 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:43.593+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:44.053 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:44.049+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:44.554 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:44.551+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:45.028 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:45.024+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:45.556 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:45.553+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:46.076 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:46.073+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:46.562 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:46.558+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:47.115 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:47.111+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:47.597 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:47.593+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:48.109 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:48.105+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:48.639 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:48.635+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:49.131 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:49.127+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:49.590 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:49.587+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:50.081 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:50.078+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:50.555 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:50.551+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:51.085 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:51.081+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:51.525 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:51.522+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:52.129 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:52.125+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:52.547 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:52.543+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:53.080 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:53.077+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:53.536 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:53.532+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:54.083 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:54.079+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:54.585 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:54.581+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:55.072 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:55.068+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:55.565 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:55.561+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:56.067 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:56.063+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:56.591 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:56.587+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:57.047 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:57.044+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:57.630 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:57.626+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:58.025 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:58.023+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:58.641 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:58.639+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:59.057 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:29:59.054+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:29:59.636 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:29:59.633+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:00.060 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:00.057+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:00.613 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:00.610+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:01.094 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:01.091+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:01.641 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:01.639+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:02.064 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:02.061+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:02.685 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:02.682+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:03.092 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:03.090+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:03.675 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:03.673+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:04.105 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:04.103+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:04.639 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:04.637+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:05.081 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:05.079+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:05.670 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:05.667+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:06.073 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:06.070+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:06.670 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:06.667+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:07.078 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:07.075+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:07.675 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:07.673+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:08.121 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:08.118+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:08.696 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:08.694+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:09.083 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:09.081+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:09.703 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:09.701+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:10.063 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:10.060+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:10.742 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:10.739+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:11.037 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:11.034+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:11.727 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:11.725+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:12.057 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:12.055+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:12.751 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:12.748+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:13.071 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:13.069+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:13.751 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:13.749+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:14.121 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:14.119+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:14.714 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:14.712+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:15.106 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:15.103+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:15.716 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:15.714+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:16.097 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:16.094+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:16.757 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:16.753+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:17.050 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:17.046+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:17.762 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:17.758+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:18.003 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:17.999+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:18.718 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:18.712+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:19.022 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:19.018+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:19.697 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:19.694+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:20.013 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:20.009+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:20.713 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:20.710+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:21.011 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:21.007+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:21.685 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:21.681+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:22.054 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:22.050+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:22.652 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:22.649+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:23.098 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:23.094+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:23.619 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:23.616+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:24.147 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:24.144+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:24.584 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:24.580+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:25.193 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:25.189+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:25.559 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:25.556+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:26.175 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:26.171+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:26.561 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:26.558+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:27.166 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:27.163+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:27.593 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:27.590+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:28.139 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:28.135+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:28.575 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:28.571+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:29.160 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:29.156+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:29.624 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:29.620+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:30.188 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:30.184+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:30.588 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:30.585+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:31.200 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:31.197+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:31.540 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:31.536+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:32.154 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:32.151+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:32.559 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:32.555+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:33.181 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:33.178+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:33.532 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:33.529+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:34.170 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:34.166+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:34.526 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:34.522+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:35.190 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:35.186+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:35.546 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:35.540+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:36.154 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:36.150+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:36.562 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:36.559+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:37.166 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:37.162+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:37.529 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:37.526+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:38.162 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:38.159+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:38.485 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:38.482+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:39.133 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:39.129+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:39.519 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:39.515+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:40.093 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:40.090+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:40.544 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:40.540+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:41.141 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:41.137+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:41.553 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:41.549+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:42.146 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:42.143+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:42.576 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:42.572+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:43.171 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:43.168+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:43.586 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:43.581+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:44.185 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:44.181+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:44.635 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:44.632+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:45.144 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:45.141+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:45.617 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:45.614+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:46.116 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:46.113+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:46.625 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:46.622+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:47.153 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:47.149+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:47.593 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:47.590+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:48.194 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:48.190+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:48.632 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:48.629+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:49.192 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:49.188+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:49.609 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:49.605+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:50.234 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:50.230+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:50.560 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:50.557+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:51.283 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:51.281+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:51.528 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:51.526+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:52.236 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:52.233+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:52.551 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:52.549+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:53.197 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:53.195+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:53.538 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:53.535+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:54.160 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:54.158+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:54.498 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:54.495+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:55.167 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:55.165+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:55.528 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:55.525+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:56.202 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:56.199+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:56.547 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:56.544+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:57.220 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:57.217+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:57.503 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:57.501+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:58.208 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:58.206+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:58.521 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:58.519+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:59.212 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:30:59.209+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:30:59.484 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:30:59.481+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:00.176 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:00.173+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:00.485 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:00.483+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:01.166 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:01.163+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:01.533 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:01.530+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:02.151 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:02.149+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:02.500 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:02.497+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:03.200 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:03.197+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:03.525 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:03.522+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:04.212 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:04.209+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:04.534 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:04.531+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:05.183 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:05.181+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:05.517 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:05.514+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:06.227 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:06.225+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:06.488 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:06.485+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:07.261 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:07.258+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:07.521 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:07.519+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:08.301 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:08.299+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:08.515 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:08.512+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:09.257 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:09.253+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:09.525 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:09.521+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:10.217 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:10.214+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:10.563 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:10.560+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:11.201 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:11.198+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:11.604 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:11.601+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:12.215 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:12.211+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:12.618 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:12.615+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:13.182 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:13.179+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:13.624 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:13.620+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:14.168 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:14.164+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:14.667 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:14.664+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:15.160 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:15.157+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:15.646 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:15.642+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:16.178 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:16.175+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:16.647 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:16.644+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:17.176 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:17.172+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:17.656 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:17.652+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:18.143 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:18.139+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:18.629 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:18.625+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:19.101 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:19.097+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:19.648 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:19.644+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:20.091 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:20.088+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:20.649 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:20.646+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:21.119 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:21.116+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:21.699 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:21.696+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:22.149 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:22.146+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:22.710 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:22.706+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:23.140 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:23.136+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:23.724 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:23.720+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:24.169 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:24.166+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:24.757 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:24.754+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:25.128 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:25.124+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:25.717 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:25.714+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:26.115 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:26.112+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:26.753 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:26.750+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:27.165 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:27.162+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:27.719 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:27.716+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:28.180 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:28.177+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:28.763 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:28.759+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:29.203 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:29.200+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:29.806 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:29.802+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:30.178 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:30.175+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:30.828 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:30.824+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:31.203 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:31.200+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:31.801 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:31.798+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:32.182 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:32.178+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:32.757 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:32.753+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:33.183 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:33.180+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:33.725 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:33.722+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:34.150 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:34.146+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:34.691 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:34.688+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:35.145 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:35.141+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:35.706 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:35.703+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:36.113 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:36.109+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:36.741 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:36.738+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:37.098 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:37.095+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:37.696 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:37.693+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:38.135 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:38.131+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:38.708 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:38.704+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:39.173 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:39.169+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:39.757 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:39.754+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:40.217 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:40.213+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:40.771 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:40.767+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:41.241 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:41.237+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:41.801 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:41.797+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:42.233 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:42.230+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:42.833 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:42.830+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:43.216 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:43.213+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:43.787 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:43.785+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:44.241 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:44.238+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:44.800 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:44.798+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:45.248 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:45.245+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:45.790 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:45.788+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:46.271 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:46.269+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:46.821 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:46.819+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:47.288 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:47.285+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:47.871 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:47.869+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:48.255 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:48.252+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:48.843 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:48.841+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:49.257 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:49.255+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:49.850 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:49.847+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:50.256 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:50.253+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:50.812 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:50.809+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:51.245 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:51.242+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:51.826 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:51.823+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:52.271 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:52.269+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:52.846 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:52.843+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:53.224 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:53.222+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:53.891 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:53.888+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:54.217 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:54.215+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:54.937 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:54.933+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:55.250 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:55.247+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:55.967 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:55.964+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:56.275 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:56.272+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:56.997 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:56.994+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:57.251 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:57.249+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:57.967 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:57.964+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:58.282 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:58.280+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:58.987 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:31:58.985+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:31:59.311 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:31:59.309+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:00.025 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:00.022+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:00.358 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:00.355+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:01.033 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:01.029+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:01.347 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:01.344+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:01.985 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:01.982+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:02.313 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:02.310+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:03.027 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:03.024+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:03.323 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:03.320+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:04.071 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:04.068+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:04.326 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:04.323+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:05.088 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:05.084+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:05.374 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:05.370+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:06.137 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:06.134+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:06.400 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:06.396+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:07.127 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:07.123+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:07.378 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:07.375+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:08.171 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:08.168+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:08.373 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:08.369+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:09.142 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:09.139+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:09.388 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:09.385+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:10.137 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:10.134+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:10.366 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:10.363+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:11.170 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:11.166+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:11.368 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:11.364+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:12.201 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:12.198+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:12.377 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:12.372+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:13.198 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:13.195+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:13.380 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:13.376+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:14.169 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:14.166+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:14.407 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:14.404+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:15.150 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:15.146+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:15.396 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:15.392+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:16.127 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:16.124+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:16.439 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:16.435+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:17.108 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:17.104+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:17.437 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:17.434+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:18.133 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:18.129+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:18.470 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:18.466+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:19.165 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:19.160+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:19.478 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:19.475+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:20.200 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:20.197+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:20.508 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:20.504+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:21.166 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:21.162+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:21.557 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:21.553+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:22.196 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:22.192+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:22.513 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:22.509+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:23.195 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:23.192+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:23.503 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:23.500+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:24.246 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:24.242+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:24.478 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:24.474+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:25.211 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:25.208+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:25.436 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:25.432+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:26.249 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:26.246+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:26.448 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:26.444+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:27.216 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:27.212+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:27.438 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:27.435+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:28.201 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:28.198+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:28.392 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:28.388+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:29.153 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:29.150+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:29.373 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:29.369+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:30.140 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:30.137+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:30.423 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:30.416+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:31.174 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:31.170+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:31.441 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:31.436+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:32.164 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:32.160+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:32.479 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:32.475+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:33.163 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:33.160+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:33.523 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:33.520+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:34.145 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:34.143+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:34.506 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:34.504+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:35.121 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:35.119+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:35.485 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:35.482+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:36.123 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:36.120+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:36.471 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:36.469+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:37.161 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:37.158+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:37.485 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:37.482+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:38.153 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:38.151+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:38.489 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:38.487+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:39.107 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:39.104+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:39.478 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:39.476+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:40.151 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:40.149+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:40.527 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:40.525+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:41.150 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:41.148+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:41.568 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:41.565+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:42.160 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:42.158+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:42.618 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:42.616+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:43.184 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:43.181+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:43.578 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:43.575+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:44.200 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:44.197+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:44.626 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:44.623+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:45.152 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:45.150+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:45.617 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:45.614+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:46.134 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:46.132+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:46.621 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:46.619+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:47.088 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:47.086+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:47.594 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:47.591+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:48.122 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:48.119+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:48.630 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:48.628+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:49.140 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:49.138+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:49.671 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:49.669+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:50.096 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:50.093+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:50.635 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:50.632+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:51.072 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:51.070+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:51.610 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:51.608+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:52.040 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:52.037+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:52.595 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:52.592+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:53.001 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:52.997+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:53.645 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:53.641+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:54.040 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:54.036+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:54.651 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:54.648+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:55.009 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:55.006+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:55.617 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:55.614+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:55.970 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:55.966+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:56.577 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:56.574+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:57.020 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:57.017+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:57.597 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:57.594+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:58.056 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:58.052+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:58.573 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:58.570+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:59.054 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:32:59.050+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:32:59.544 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:32:59.540+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:00.084 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:00.080+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:00.498 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:00.494+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:01.081 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:01.078+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:01.485 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:01.482+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:02.107 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:02.104+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:02.456 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:02.453+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:03.092 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:03.088+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:03.416 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:03.412+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:04.081 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:04.077+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:04.421 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:04.418+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:05.048 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:05.045+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:05.418 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:05.415+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:06.014 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:06.010+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:06.445 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:06.442+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:06.969 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:06.966+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:07.479 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:07.476+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:07.943 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:07.940+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:08.515 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:08.512+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:08.989 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:08.986+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:09.503 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:09.500+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:09.970 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:09.967+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:10.490 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:10.487+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:10.927 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:10.924+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:11.527 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:11.523+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:11.942 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:11.939+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:12.497 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:12.494+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:12.949 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:12.945+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:13.454 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:13.450+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:13.917 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:13.914+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:14.463 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:14.460+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:14.893 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:14.890+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:15.480 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:15.476+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:15.920 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:15.916+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:16.511 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:16.508+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:16.884 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:16.880+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:17.505 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:17.502+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:17.876 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:17.872+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:18.515 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:18.512+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:18.839 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:18.836+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:19.521 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:19.518+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:19.855 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:19.852+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:20.501 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:20.497+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:20.835 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:20.831+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:21.518 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:21.515+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:21.845 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:21.842+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:22.485 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:22.481+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:22.848 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:22.845+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:23.486 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:23.482+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:23.835 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:23.831+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:24.455 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:24.452+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:24.832 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:24.829+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:25.484 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:25.481+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:25.869 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:25.866+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:26.478 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:26.475+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:26.861 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:26.858+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:27.518 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:27.515+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:27.895 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:27.892+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:28.504 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:28.501+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:28.939 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:28.937+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:29.534 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:29.532+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:29.953 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:29.950+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:30.510 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:30.507+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:30.940 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:30.938+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:31.493 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:31.491+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:31.902 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:31.900+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:32.510 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:32.507+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:32.875 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:32.872+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:33.480 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:33.477+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:33.912 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:33.910+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:34.479 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:34.476+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:34.888 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:34.885+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:35.456 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:35.454+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:35.849 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:35.846+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:36.492 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:36.490+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:36.829 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:36.826+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:37.528 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:37.526+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:37.842 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:37.840+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:38.491 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:38.488+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:38.874 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:38.872+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:39.446 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:39.444+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:39.841 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:39.839+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:40.416 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:40.414+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:40.848 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:40.846+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:41.415 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:41.413+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:41.839 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:41.837+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:42.410 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:42.408+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:42.804 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:42.802+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:43.432 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:43.430+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:43.814 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:43.812+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:44.410 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:44.406+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:44.842 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:44.838+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:45.404 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:45.401+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:45.811 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:45.807+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:46.443 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:46.440+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:46.789 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:46.785+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:47.396 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:47.393+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:47.831 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:47.827+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:48.442 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:48.439+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:48.782 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:48.779+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:49.464 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:49.460+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:49.737 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:49.733+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:50.496 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:50.493+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:50.786 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:50.782+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:51.484 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:51.481+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:51.784 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:51.781+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:52.465 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:52.461+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:52.802 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:52.796+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:53.496 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:53.492+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:53.814 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:53.811+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:54.472 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:54.469+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:54.843 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:54.839+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:55.465 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:55.459+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:55.822 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:55.819+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:56.456 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:56.453+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:56.866 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:56.862+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:57.493 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:57.490+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:57.828 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:57.824+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:58.539 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:58.535+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:58.870 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:58.866+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:59.491 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:33:59.488+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:33:59.854 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:33:59.851+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:00.515 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:00.511+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:00.895 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:00.892+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:01.546 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:01.542+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:01.905 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:01.901+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:02.595 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:02.592+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:02.924 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:02.921+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:03.622 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:03.618+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:03.911 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:03.908+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:04.590 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:04.587+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:04.901 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:04.897+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:05.558 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:05.555+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:05.872 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:05.869+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:06.581 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:06.578+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:06.897 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:06.894+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:07.564 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:07.561+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:07.909 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:07.906+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:08.568 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:08.564+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:08.954 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:08.951+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:09.020 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:34:09.016+0000 7fd64ac56700 0 -- 172.21.15.82:0/3601686610 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x5627827aeb20 msgr2=0x7fd63004f710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-18T16:34:09.572 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:09.569+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:09.864 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T16:34:09.860+0000 7fd64b457700 0 -- 172.21.15.82:0/3601686610 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x5627827aeb20 msgr2=0x7fd63004f710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-18T16:34:09.932 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:09.928+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:10.587 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:10.584+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:10.900 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:10.897+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:11.542 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:11.539+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:11.942 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:11.938+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:12.535 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:12.532+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:12.919 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:12.915+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:13.556 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:13.552+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:13.915 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:13.912+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:14.538 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:14.534+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:14.944 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:14.940+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:15.493 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:15.489+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:15.921 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:15.918+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:16.452 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:16.448+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:16.949 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:16.945+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:17.451 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:17.448+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:17.966 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:17.964+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:18.452 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:18.450+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:18.991 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:18.988+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:19.429 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:19.426+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:19.967 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:19.964+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:20.465 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:20.463+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:20.996 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:20.993+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:21.425 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:21.423+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:21.957 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:21.954+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:22.426 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:22.424+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:23.004 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:23.002+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:23.426 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:23.424+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:24.045 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:24.043+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:24.417 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:24.414+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:25.084 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:25.082+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:25.416 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:25.414+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:26.096 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:26.094+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:26.430 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:26.428+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:27.069 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:27.067+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:27.438 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:27.436+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:28.041 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:28.039+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:28.438 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:28.436+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:29.018 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:29.015+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:29.414 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:29.412+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:30.004 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:30.002+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:30.420 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:30.418+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:31.022 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:31.020+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:31.427 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:31.424+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:32.027 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:32.025+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:32.460 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:32.457+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:33.057 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:33.055+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:33.505 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:33.502+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:34.104 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:34.102+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:34.519 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:34.517+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:35.116 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:35.114+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:35.554 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:35.552+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:36.131 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:36.128+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:36.534 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:36.531+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:37.132 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:37.128+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:37.528 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:37.525+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:38.168 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:38.165+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:38.552 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:38.549+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:39.129 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:39.126+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:39.578 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:39.574+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:40.106 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:40.102+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:40.555 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:40.552+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:41.131 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:41.127+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:41.576 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:41.573+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:42.110 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:42.107+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:42.544 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:42.541+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:43.120 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:43.117+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:43.542 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:43.539+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:44.095 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:44.092+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:44.591 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:44.588+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:45.054 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:45.051+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:45.609 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:45.606+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:46.094 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:46.091+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:46.596 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:46.593+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:47.053 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:47.049+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:47.570 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:47.566+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:48.059 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:48.056+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:48.600 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:48.597+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:49.028 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:49.025+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:49.604 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:49.601+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:50.016 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:50.013+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:50.560 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:50.557+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:51.057 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:51.053+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:51.608 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:51.605+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:52.016 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:52.012+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:52.638 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:52.635+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:53.036 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:53.033+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:53.630 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:53.627+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:54.056 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:54.053+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:54.584 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:54.581+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:55.022 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:55.019+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:55.550 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:55.547+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:56.069 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:56.066+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:56.557 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:56.553+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:57.029 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:57.026+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:57.520 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:57.516+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:58.062 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:58.059+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:58.523 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:58.520+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:59.108 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:34:59.105+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:34:59.557 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:34:59.554+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:00.092 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:00.089+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:00.594 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:00.591+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:01.095 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:01.091+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:01.594 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:01.591+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:02.146 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:02.142+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:02.636 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:02.632+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:03.179 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:03.176+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:03.686 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:03.683+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:04.179 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:04.176+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:04.639 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:04.635+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:05.164 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:05.161+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:05.667 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:05.663+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:06.168 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:06.165+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:06.635 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:06.631+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:07.170 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:07.167+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:07.656 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:07.652+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:08.158 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:08.154+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:08.652 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:08.649+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:09.141 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:09.138+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:09.626 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:09.623+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:10.159 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:10.156+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:10.598 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:10.595+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:11.166 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:11.164+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:11.629 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:11.626+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:12.131 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:12.129+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:12.607 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:12.605+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:13.135 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:13.133+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:13.633 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:13.630+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:14.118 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:14.116+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:14.638 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:14.636+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:15.116 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:15.114+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:15.624 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:15.622+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:16.092 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:16.089+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:16.597 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:16.593+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:17.121 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:17.118+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:17.568 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:17.566+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:18.103 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:18.101+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:18.527 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:18.524+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:19.070 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:19.068+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:19.501 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:19.498+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:20.040 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:20.038+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:20.541 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:20.538+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:21.045 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:21.043+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:21.588 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:21.586+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:22.058 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:22.055+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:22.626 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:22.623+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:23.013 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:23.011+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:23.588 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:23.585+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:24.026 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:24.024+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:24.550 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:24.548+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:25.058 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:25.055+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:25.528 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:25.526+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:26.059 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:26.056+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:26.513 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:26.510+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:27.062 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:27.060+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:27.497 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:27.494+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:28.084 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:28.080+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:28.532 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:28.529+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:29.094 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:29.091+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:29.559 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:29.555+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:30.050 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:30.046+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:30.558 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:30.555+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:31.091 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:31.088+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:31.557 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:31.554+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:32.124 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:32.121+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:32.514 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:32.510+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:33.170 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:33.166+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:33.466 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:33.462+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:34.217 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:34.213+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:34.513 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:34.509+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:35.170 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:35.167+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:35.515 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:35.512+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:36.162 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:36.158+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:36.475 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:36.472+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:37.165 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:37.162+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:37.447 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:37.444+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:38.152 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:38.149+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:38.482 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:38.479+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:39.113 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:39.109+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:39.479 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:39.475+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:40.144 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:40.141+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:40.514 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:40.511+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:41.157 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:41.153+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:41.512 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:41.509+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:42.196 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:42.193+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:42.496 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:42.493+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:43.212 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:43.209+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:43.478 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:43.475+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:44.226 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:44.223+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:44.491 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:44.487+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:45.208 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:45.205+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:45.505 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:45.501+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:46.223 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:46.220+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:46.502 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:46.499+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:47.261 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:47.258+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:47.538 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:47.535+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:48.238 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:48.235+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:48.546 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:48.543+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:49.208 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:49.205+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:49.581 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:49.577+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:50.166 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:50.162+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:50.569 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:50.566+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:51.206 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:51.203+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:51.592 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:51.589+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:52.184 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:52.181+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:52.590 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:52.587+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:53.148 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:53.144+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:53.572 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:53.569+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:54.160 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:54.156+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:54.537 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:54.533+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:55.138 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:55.135+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:55.507 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:55.503+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:56.089 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:56.086+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:56.517 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:56.513+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:57.091 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:57.088+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:57.471 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:57.467+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:58.064 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:58.061+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:58.502 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:58.499+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:59.052 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:35:59.048+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:35:59.508 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:35:59.505+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:00.046 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:00.043+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:00.476 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:00.473+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:01.040 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:01.037+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:01.458 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:01.455+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:02.051 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:02.048+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:02.439 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:02.437+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:03.086 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:03.084+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:03.450 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:03.447+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:04.133 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:04.131+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:04.467 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:04.465+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:05.176 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:05.174+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:05.501 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:05.499+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:06.173 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:06.170+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:06.512 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:06.509+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:07.164 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:07.161+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:07.491 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:07.485+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:08.162 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:08.152+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:08.496 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:08.494+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:09.143 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:09.141+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:09.479 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:09.477+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:10.127 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:10.125+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:10.450 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:10.447+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:11.098 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:11.096+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:11.433 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:11.430+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:12.071 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:12.069+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:12.433 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:12.431+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:13.091 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:13.088+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:13.447 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:13.445+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:14.093 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:14.090+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:14.473 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:14.471+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:15.114 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:15.111+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:15.506 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:15.503+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:16.161 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:16.159+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:16.493 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:16.491+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:17.161 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:17.159+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:17.515 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:17.513+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:18.154 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:18.152+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:18.517 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:18.515+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:19.196 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:19.193+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:19.551 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:19.548+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:20.169 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:20.166+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:20.554 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:20.550+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:21.195 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:21.192+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:21.512 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:21.508+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:22.223 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:22.220+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:22.530 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:22.527+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:23.255 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:23.252+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:23.529 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:23.526+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:24.280 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:24.277+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:24.482 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:24.479+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:25.270 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:25.266+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:25.477 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:25.474+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:26.311 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:26.307+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:26.515 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:26.511+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:27.278 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:27.275+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:27.484 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:27.481+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:28.289 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:28.286+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:28.521 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:28.517+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:29.293 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:29.290+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:29.511 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:29.508+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:30.281 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:30.277+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:30.517 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:30.514+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:31.270 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:31.267+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:31.562 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:31.558+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:32.242 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:32.238+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:32.604 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:32.600+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:33.201 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:33.198+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:33.600 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:33.596+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:34.171 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:34.167+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:34.561 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:34.558+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:35.176 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:35.172+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:35.553 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:35.550+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:36.187 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:36.183+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:36.547 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:36.544+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:37.200 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:37.197+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:37.587 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:37.583+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:38.227 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:38.224+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:38.617 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:38.614+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:39.196 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:39.192+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:39.655 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:39.652+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:40.216 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:40.213+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:40.674 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:40.671+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:41.221 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:41.218+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:41.669 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:41.666+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:42.238 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:42.235+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:42.648 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:42.645+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:43.221 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:43.217+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:43.624 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:43.621+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:44.249 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:44.246+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:44.577 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:44.574+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:45.263 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:45.260+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:45.557 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:45.554+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:46.221 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:46.218+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:46.601 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:46.598+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:47.244 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:47.241+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:47.606 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:47.602+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:48.255 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:48.252+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:48.594 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:48.591+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:49.271 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:49.267+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:49.570 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:49.567+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:50.287 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:50.284+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:50.543 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:50.540+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:51.267 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:51.264+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:51.561 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:51.558+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:52.274 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:52.271+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:52.597 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:52.593+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:53.241 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:53.238+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:53.617 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:53.614+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:54.230 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:54.228+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:54.569 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:54.567+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:55.274 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:55.271+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:55.574 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:55.571+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:56.303 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:56.301+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:56.543 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:56.541+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:57.337 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:57.335+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:57.510 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:57.507+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:58.342 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:58.339+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:58.548 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:58.546+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:59.358 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:36:59.355+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:36:59.529 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:36:59.527+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:00.335 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:00.333+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:00.535 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:00.532+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:01.354 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:01.352+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:01.510 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:01.507+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:02.317 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:02.315+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:02.515 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:02.513+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:03.281 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:03.279+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:03.468 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:03.465+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:04.298 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:04.296+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:04.437 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:04.435+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:05.336 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:05.333+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:05.476 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:05.474+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:06.315 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:06.312+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:06.499 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:06.496+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:07.330 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:07.328+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:07.510 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:07.507+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:08.353 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:08.351+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:08.545 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:08.543+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:09.326 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:09.323+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:09.496 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:09.493+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:10.372 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:10.369+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:10.474 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:10.471+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:11.354 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:11.352+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:11.437 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:11.435+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:12.395 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:12.392+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:12.457 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:12.454+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:13.384 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:13.381+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:13.417 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:13.414+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:14.429 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:14.426+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:14.451 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:14.448+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:15.475 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:15.472+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:15.482 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:15.479+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:16.469 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:16.466+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:16.511 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:16.508+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:17.479 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:17.476+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:17.532 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:17.529+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:18.440 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:18.437+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:18.487 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:18.484+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:19.445 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:19.442+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:19.489 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:19.486+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:20.432 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:20.429+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:20.476 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:20.472+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:21.449 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:21.446+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:21.467 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:21.464+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:22.450 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:22.447+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:22.504 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:22.500+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:23.469 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:23.466+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:23.489 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:23.486+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:24.446 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:24.443+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:24.535 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:24.532+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:25.493 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:25.490+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:25.494 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:25.491+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:26.473 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:26.470+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:26.488 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:26.484+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:27.449 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:27.446+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:27.521 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:27.518+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:28.431 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:28.428+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:28.567 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:28.564+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:29.450 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:29.446+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:29.541 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:29.538+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:30.476 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:30.473+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:30.560 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:30.557+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:31.449 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:31.446+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:31.531 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:31.528+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:32.404 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:32.401+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:32.531 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:32.528+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:33.400 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:33.397+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:33.490 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:33.487+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:34.389 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:34.386+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:34.525 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:34.522+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:35.398 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:35.395+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:35.500 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:35.497+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:36.359 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:36.356+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:36.525 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:36.522+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:37.331 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:37.328+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:37.489 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:37.486+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:38.357 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:38.354+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:38.518 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:38.514+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:39.347 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:39.344+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:39.547 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:39.543+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:40.371 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:40.367+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:40.549 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:40.546+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:41.356 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:41.352+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:41.561 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:41.557+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:42.337 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:42.334+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:42.591 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:42.588+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:43.340 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:43.337+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:43.598 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:43.595+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:44.331 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:44.328+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:44.598 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:44.594+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:45.357 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:45.354+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:45.567 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:45.564+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:46.311 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:46.309+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:46.571 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:46.569+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:47.273 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:47.270+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:47.601 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:47.598+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:48.299 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:48.296+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:48.643 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:48.641+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:49.261 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:49.259+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:49.690 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:49.688+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:50.221 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:50.218+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:50.655 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:50.653+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:51.212 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:51.210+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:51.667 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:51.664+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:52.214 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:52.211+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:52.682 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:52.679+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:53.199 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:53.196+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:53.651 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:53.649+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:54.203 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:54.201+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:54.609 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:54.607+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:55.167 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:55.165+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:55.624 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:55.622+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:56.179 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:56.177+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:56.653 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:56.651+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:57.185 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:57.182+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:57.693 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:57.690+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:58.181 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:58.178+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:58.707 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:58.704+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:59.210 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:37:59.207+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:37:59.728 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:37:59.726+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:00.240 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:00.237+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:00.737 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:00.735+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:01.285 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:01.283+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:01.752 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:01.749+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:02.332 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:02.329+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:02.712 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:02.709+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:03.349 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:03.347+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:03.743 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:03.741+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:04.321 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:04.318+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:04.735 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:04.732+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:05.355 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:05.352+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:05.776 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:05.772+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:06.390 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:06.386+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:06.727 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:06.724+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:07.392 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:07.389+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:07.716 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:07.712+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:08.433 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:08.430+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:08.740 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:08.737+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:09.452 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:09.448+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:09.790 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:09.787+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:10.489 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:10.485+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:10.755 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:10.752+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:11.523 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:11.520+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:11.777 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:11.774+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:12.569 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:12.566+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:12.797 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:12.794+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:13.567 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:13.563+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:13.811 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:13.808+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:14.521 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:14.518+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:14.791 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:14.788+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:15.513 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:15.510+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:15.828 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:15.825+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:16.487 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:16.484+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:16.859 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:16.856+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:17.470 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:17.467+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:17.855 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:17.852+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:18.505 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:18.502+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:18.863 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:18.859+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:19.466 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:19.463+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:19.846 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:19.843+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:20.495 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:20.491+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:20.872 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:20.868+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:21.483 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:21.479+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:21.857 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:21.854+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:22.512 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:22.509+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:22.846 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:22.843+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:23.489 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:23.485+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:23.862 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:23.859+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:24.446 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:24.442+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:24.833 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:24.829+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:25.495 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:25.492+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:25.787 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:25.783+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:26.482 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:26.479+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:26.752 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:26.749+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:27.483 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:27.480+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:27.751 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:27.748+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:28.533 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:28.529+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:28.794 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:28.791+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:29.522 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:29.519+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:29.790 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:29.787+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:30.563 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:30.560+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:30.824 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:30.821+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:31.595 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:31.591+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:31.844 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:31.840+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:32.597 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:32.594+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:32.815 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:32.811+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:33.639 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:33.636+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:33.770 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:33.767+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:34.595 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:34.591+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:34.727 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:34.723+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:35.617 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:35.614+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:35.730 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:35.727+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:36.646 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:36.642+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:36.778 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:36.775+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:37.613 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:37.610+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:37.785 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:37.782+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:38.648 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:38.646+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:38.778 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:38.775+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:39.681 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:39.678+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:39.775 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:39.773+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:40.654 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:40.652+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:40.738 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:40.735+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:41.661 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:41.658+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:41.730 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:41.727+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:42.636 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:42.633+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:42.742 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:42.740+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:43.655 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:43.652+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:43.766 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:43.764+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:44.674 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:44.671+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:44.790 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:44.788+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:45.699 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:45.697+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:45.827 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:45.824+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:46.680 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:46.677+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:46.852 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:46.849+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:47.707 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:47.705+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:47.848 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:47.845+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:48.702 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:48.699+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:48.867 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:48.864+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:49.694 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:49.692+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:49.896 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:49.893+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:50.679 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:50.676+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:50.937 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:50.935+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:51.718 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:51.716+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:51.950 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:51.948+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:52.764 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:52.761+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:52.964 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:52.962+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:53.787 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:53.784+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:53.918 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:53.915+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:54.827 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:54.825+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:54.882 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:54.880+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:55.825 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:55.822+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:55.906 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:55.903+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:56.779 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:56.776+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:56.954 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:56.951+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:57.750 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:57.747+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:58.002 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:57.999+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:58.759 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:58.756+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:59.027 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:59.023+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:59.740 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:38:59.737+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:38:59.981 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:38:59.978+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:00.705 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:00.701+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:01.028 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:01.024+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:01.666 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:01.663+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:02.045 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:02.041+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:02.697 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:02.693+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:03.069 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:03.066+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:03.650 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:03.646+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:04.080 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:04.077+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:04.684 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:04.681+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:05.102 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:05.099+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:05.650 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:05.646+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:06.069 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:06.066+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:06.690 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:06.687+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:07.056 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:07.053+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:07.717 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:07.714+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:08.068 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:08.064+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:08.693 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:08.689+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:09.054 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:09.050+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:09.698 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:09.695+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:10.012 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:10.009+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:10.664 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:10.661+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:11.025 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:11.022+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:11.646 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:11.643+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:12.058 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:12.053+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:12.609 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:12.606+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:13.021 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:13.018+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:13.651 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:13.647+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:14.002 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:13.999+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:14.607 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:14.604+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:14.977 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:14.974+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:15.607 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:15.604+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:16.003 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:16.000+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:16.630 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:16.627+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:17.010 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:17.007+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:17.670 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:17.667+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:18.053 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:18.050+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:18.657 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:18.654+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:19.068 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:19.065+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:19.614 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:19.611+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:20.059 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:20.056+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:20.567 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:20.564+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:21.105 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:21.101+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:21.586 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:21.583+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:22.149 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:22.146+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:22.604 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:22.601+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:23.142 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:23.139+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:23.556 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:23.553+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:24.170 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:24.165+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:24.559 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:24.556+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:25.178 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:25.175+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:25.556 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:25.552+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:26.193 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:26.190+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:26.597 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:26.594+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:27.207 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:27.203+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:27.562 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:27.558+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:28.184 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:28.180+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:28.518 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:28.515+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:29.177 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:29.174+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:29.489 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:29.483+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:30.147 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:30.144+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:30.507 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:30.505+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:31.122 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:31.119+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:31.515 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:31.513+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:32.092 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:32.090+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:32.535 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:32.532+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:33.107 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:33.105+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:33.538 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:33.535+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:34.115 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:34.113+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:34.503 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:34.500+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:35.095 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:35.093+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:35.511 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:35.509+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:36.130 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:36.127+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:36.489 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:36.487+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:37.118 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:37.115+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:37.513 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:37.511+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:38.147 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:38.145+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:38.530 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:38.527+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:39.185 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:39.183+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:39.525 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:39.522+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:40.141 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:40.139+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:40.541 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:40.538+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:41.127 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:41.125+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:41.502 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:41.499+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:42.125 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:42.123+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:42.459 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:42.456+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:43.131 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:43.128+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:43.489 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:43.487+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:44.162 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:44.160+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:44.503 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:44.501+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:45.150 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:45.148+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:45.521 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:45.519+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:46.154 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:46.151+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:46.532 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:46.529+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:47.130 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:47.127+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:47.502 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:47.499+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:48.142 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:48.138+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:48.522 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:48.519+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:49.124 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:49.121+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:49.540 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:49.537+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:50.153 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:50.150+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:50.539 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:50.536+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:51.137 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:51.134+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:51.517 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:51.514+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:52.171 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:52.168+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:52.562 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:52.558+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:53.132 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:53.128+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:53.519 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:53.515+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:54.100 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:54.097+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:54.497 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:54.494+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:55.062 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:55.059+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:55.455 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:55.452+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:56.039 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:56.036+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:56.451 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:56.448+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:57.014 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:57.011+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:57.427 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:57.424+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:58.011 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:58.008+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:58.383 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:58.380+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:59.033 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:59.030+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:59.427 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:39:59.423+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:39:59.994 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:39:59.991+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:00.407 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:00.404+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:00.970 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:00.967+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:01.435 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:01.431+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:01.958 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:01.955+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:02.388 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:02.385+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:03.005 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:03.002+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:03.362 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:03.358+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:03.983 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:03.980+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:04.313 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:04.309+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:05.007 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:05.004+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:05.346 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:05.343+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:06.057 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:06.054+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:06.395 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:06.392+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:07.091 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:07.087+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:07.434 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:07.431+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:08.125 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:08.122+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:08.484 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:08.481+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:09.148 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:09.145+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:09.531 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:09.528+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:10.159 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:10.156+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:10.559 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:10.556+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:11.113 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:11.110+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:11.518 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:11.515+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:12.151 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:12.148+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:12.549 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:12.545+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:13.200 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:13.197+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:13.565 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:13.562+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:14.195 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:14.192+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:14.565 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:14.562+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:15.211 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:15.208+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:15.536 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:15.533+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:16.180 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:16.177+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:16.494 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:16.491+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:17.178 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:17.175+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:17.529 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:17.526+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:18.215 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:18.212+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:18.511 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:18.508+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:19.209 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:19.206+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:19.557 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:19.554+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:20.192 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:20.189+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:20.583 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:20.580+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:21.237 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:21.234+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:21.621 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:21.618+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:22.253 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:22.251+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:22.573 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:22.571+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:23.245 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:23.243+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:23.548 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:23.546+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:24.227 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:24.225+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:24.612 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:24.586+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:25.271 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:25.269+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:25.539 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:25.537+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:26.300 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:26.298+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:26.540 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:26.537+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:27.302 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:27.299+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:27.498 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:27.495+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:28.291 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:28.289+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:28.524 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:28.521+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:29.243 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:29.241+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:29.493 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:29.491+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:30.247 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:30.244+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:30.539 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:30.537+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:31.283 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:31.281+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:31.507 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:31.505+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:32.282 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:32.279+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:32.459 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:32.457+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:33.237 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:33.234+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:33.435 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:33.432+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:34.245 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:34.243+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:34.430 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:34.427+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:35.265 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:35.263+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:35.445 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:35.443+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:36.263 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:36.261+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:36.478 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:36.476+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:37.268 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:37.266+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:37.483 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:37.481+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:38.220 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:38.218+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:38.447 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:38.445+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:39.248 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:39.245+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:39.467 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:39.465+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:40.248 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:40.245+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:40.476 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:40.473+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:41.262 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:41.259+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:41.460 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:41.456+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:42.252 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:42.249+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:42.448 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:42.444+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:43.231 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:43.228+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:43.473 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:43.470+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:44.213 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:44.210+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:44.479 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:44.475+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:45.221 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:45.218+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:45.449 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:45.446+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:46.235 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:46.232+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:46.435 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:46.431+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:47.211 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:47.208+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:47.421 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:47.417+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:48.177 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:48.174+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:48.420 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:48.416+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:49.193 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:49.190+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:49.406 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:49.403+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:50.148 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:50.145+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:50.383 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:50.379+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:51.113 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:51.110+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:51.337 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:51.334+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:52.086 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:52.083+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:52.323 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:52.320+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:53.049 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:53.046+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:53.374 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:53.371+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:54.006 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:54.003+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:54.389 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:54.386+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:55.029 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:55.026+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:55.356 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:55.353+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:55.996 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:55.992+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:56.339 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:56.335+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:56.975 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:56.972+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:57.386 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:57.383+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:57.936 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:57.933+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:58.427 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:58.424+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:58.911 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:58.908+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:59.394 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:40:59.391+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:40:59.883 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:40:59.880+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:00.373 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:00.370+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:00.835 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:00.831+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:01.363 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:01.359+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:01.832 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:01.828+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:02.361 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:02.358+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:02.857 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:02.854+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:03.391 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:03.388+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:03.854 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:03.851+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:04.377 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:04.374+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:04.870 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:04.867+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:05.381 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:05.378+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:05.859 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:05.855+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:06.361 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:06.357+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:06.810 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:06.806+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:07.402 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:07.399+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:07.819 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:07.816+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:08.442 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:08.439+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:08.823 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:08.820+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:09.399 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:09.395+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:09.827 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:09.823+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:10.363 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:10.360+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:10.851 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:10.848+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:11.358 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:11.355+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:11.826 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:11.823+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:12.385 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:12.382+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:12.871 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:12.868+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:13.337 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:13.333+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:13.867 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:13.864+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:14.301 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:14.299+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:14.913 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:14.911+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:15.305 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:15.303+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:15.945 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:15.943+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:16.324 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:16.322+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:16.980 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:16.977+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:17.278 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:17.276+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:18.024 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:18.021+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:18.248 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:18.246+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:19.016 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:19.013+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:19.268 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:19.266+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:20.030 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:20.028+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:20.257 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:20.254+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:21.018 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:21.016+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:21.272 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:21.269+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:22.017 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:22.014+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:22.252 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:22.250+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:23.043 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:23.038+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:23.267 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:23.265+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:23.994 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:23.991+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:24.236 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:24.234+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:24.983 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:24.981+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:25.220 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:25.218+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:25.950 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:25.948+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:26.207 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:26.205+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:26.970 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:26.968+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:27.179 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:27.177+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:27.921 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:27.918+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:28.210 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:28.208+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:28.922 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:28.920+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:29.244 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:29.242+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:29.893 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:29.891+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:30.290 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:30.287+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:30.888 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:30.886+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:31.334 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:31.331+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:31.938 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:31.936+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:32.324 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:32.321+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:32.960 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:32.957+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:33.361 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:33.358+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:33.966 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:33.963+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:34.388 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:34.385+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:34.932 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:34.928+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:35.367 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:35.364+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:35.970 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:35.967+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:36.412 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:36.408+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:36.930 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:36.927+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:37.432 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:37.428+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:37.971 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:37.968+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:38.464 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:38.461+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:39.001 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:38.998+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:39.436 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:39.433+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:40.029 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:40.025+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:40.393 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:40.389+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:40.994 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:40.991+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:41.389 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:41.386+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:41.957 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:41.954+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:42.405 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:42.402+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:42.995 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:42.992+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:43.366 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:43.362+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:44.011 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:44.007+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:44.398 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:44.394+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:44.998 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:44.995+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:45.357 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:45.354+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:46.044 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:46.040+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:46.317 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:46.314+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:47.071 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:47.068+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:47.306 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:47.302+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:48.065 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:48.062+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:48.264 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:48.261+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:49.058 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:49.055+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:49.264 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:49.261+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:50.088 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:50.084+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:50.314 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:50.311+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:51.102 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:51.099+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:51.318 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:51.314+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:52.130 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:52.127+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:52.306 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:52.303+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:53.120 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:53.117+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:53.277 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:53.274+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:54.089 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:54.086+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:54.254 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:54.251+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:55.086 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:55.083+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:55.286 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:55.283+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:56.067 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:56.063+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:56.294 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:56.291+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:57.098 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:57.095+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:57.308 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:57.305+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:58.138 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:58.135+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:58.328 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:58.325+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:59.124 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:41:59.121+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:41:59.288 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:41:59.285+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:00.160 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:00.157+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:00.242 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:00.239+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:01.158 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:01.154+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:01.200 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:01.197+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:02.168 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:02.165+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:02.230 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:02.227+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:03.152 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:03.149+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:03.220 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:03.217+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:04.142 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:04.139+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:04.206 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:04.203+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:05.176 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:05.172+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:05.244 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:05.240+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:06.193 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:06.191+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:06.249 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:06.247+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:07.221 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:07.219+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:07.232 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:07.230+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:08.194 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:08.191+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:08.242 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:08.239+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:09.150 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:09.147+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:09.262 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:09.259+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:10.114 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:10.112+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:10.246 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:10.244+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:11.068 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:11.066+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:11.204 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:11.202+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:12.104 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:12.101+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:12.202 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:12.200+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:13.128 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:13.126+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:13.212 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:13.210+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:14.082 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:14.080+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:14.237 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:14.235+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:15.064 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:15.062+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:15.196 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:15.194+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:16.101 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:16.099+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:16.172 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:16.170+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:17.152 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:17.149+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:17.212 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:17.210+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:18.179 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:18.176+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:18.191 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:18.189+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:19.161 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:19.159+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:19.229 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:19.227+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:20.156 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:20.153+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:20.229 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:20.227+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:21.174 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:21.171+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:21.270 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:21.267+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:22.167 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:22.165+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:22.268 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:22.266+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:23.129 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:23.127+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:23.255 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:23.252+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:24.101 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:24.098+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:24.243 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:24.239+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:25.084 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:25.081+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:25.238 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:25.235+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:26.036 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:26.033+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:26.256 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:26.253+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:27.031 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:27.028+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:27.297 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:27.293+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:28.024 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:28.021+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:28.263 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:28.260+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:29.048 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:29.045+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:29.271 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:29.267+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:30.025 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:30.022+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:30.260 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:30.257+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:30.995 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:30.992+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:31.290 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:31.286+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:32.023 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:32.020+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:32.294 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:32.291+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:33.040 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:33.037+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:33.276 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:33.273+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:34.051 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:34.047+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:34.291 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:34.288+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:35.097 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:35.094+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:35.316 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:35.312+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:36.135 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:36.132+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:36.324 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:36.320+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:37.114 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:37.110+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:37.358 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:37.355+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:38.100 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:38.097+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:38.406 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:38.403+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:39.099 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:39.096+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:39.364 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:39.361+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:40.143 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:40.140+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:40.361 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:40.358+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:41.189 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:41.186+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:41.406 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:41.402+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:42.222 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:42.218+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:42.410 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:42.407+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:43.261 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:43.258+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:43.379 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:43.376+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:44.234 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:44.231+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:44.393 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:44.390+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:45.196 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:45.193+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:45.367 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:45.364+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:46.209 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:46.205+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:46.330 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:46.327+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:47.217 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:47.214+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:47.374 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:47.371+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:48.198 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:48.195+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:48.334 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:48.330+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:49.188 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:49.185+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:49.286 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:49.283+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:50.177 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:50.174+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:50.320 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:50.316+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:51.148 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:51.145+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:51.321 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:51.318+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:52.174 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:52.171+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:52.359 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:52.355+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:53.177 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:53.173+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:53.319 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:53.316+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:54.212 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:54.209+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:54.360 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:54.357+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:55.260 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:55.257+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:55.404 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:55.401+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:56.305 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:56.302+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:56.435 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:56.432+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:57.301 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:57.299+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:57.484 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:57.481+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:58.342 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:58.340+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:58.471 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:58.469+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:59.354 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:42:59.352+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:42:59.449 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:42:59.447+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:00.325 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:00.323+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:00.404 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:00.402+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:01.305 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:01.303+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:01.400 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:01.398+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:02.346 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:02.344+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:02.386 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:02.384+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:03.354 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:03.352+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:03.378 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:03.376+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:04.381 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:04.379+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:04.400 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:04.398+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:05.370 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:05.368+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:05.422 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:05.420+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:06.332 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:06.330+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:06.400 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:06.398+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:07.346 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:07.343+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:07.378 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:07.376+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:08.339 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:08.337+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:08.380 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:08.373+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:09.314 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:09.312+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:09.372 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:09.370+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:10.312 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:10.310+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:10.363 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:10.361+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:11.280 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:11.278+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:11.356 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:11.354+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:12.266 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:12.264+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:12.405 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:12.403+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:13.238 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:13.236+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:13.384 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:13.382+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:14.280 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:14.277+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:14.400 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:14.398+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:15.275 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:15.273+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:15.379 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:15.377+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:16.303 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:16.300+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:16.347 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:16.344+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:17.310 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:17.307+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:17.350 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:17.346+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:18.276 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:18.273+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:18.332 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:18.329+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:19.286 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:19.283+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:19.300 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:19.297+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:20.239 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:20.236+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:20.259 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:20.256+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:21.221 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:21.217+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:21.277 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:21.274+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:22.242 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:22.239+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:22.286 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:22.283+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:23.260 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:23.257+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:23.335 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:23.332+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:24.306 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:24.303+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:24.328 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:24.325+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:25.283 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:25.280+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:25.286 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:25.283+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:26.248 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:26.245+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:26.309 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:26.306+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:27.225 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:27.221+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:27.307 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:27.304+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:28.178 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:28.175+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:28.330 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:28.327+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:29.213 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:29.209+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:29.302 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:29.298+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:30.248 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:30.245+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:30.262 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:30.259+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:31.260 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:31.257+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:31.290 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:31.287+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:32.250 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:32.247+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:32.340 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:32.337+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:33.282 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:33.278+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:33.382 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:33.379+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:34.278 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:34.275+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:34.334 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:34.331+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:35.257 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:35.254+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:35.294 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:35.290+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:36.236 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:36.233+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:36.324 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:36.321+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:37.230 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:37.227+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:37.344 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:37.341+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:38.186 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:38.183+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:38.338 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:38.335+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:39.177 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:39.174+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:39.341 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:39.338+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:40.208 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:40.204+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:40.338 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:40.335+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:41.212 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:41.209+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:41.311 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:41.308+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:42.233 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:42.230+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:42.281 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:42.278+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:43.260 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:43.257+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:43.282 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:43.279+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:44.240 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:44.237+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:44.297 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:44.294+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:45.286 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:45.283+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:45.298 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:45.294+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:46.297 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:46.294+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:46.298 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:46.295+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:47.312 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:47.309+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:47.319 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:47.316+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:48.280 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:48.277+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:48.326 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:48.323+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:49.250 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:49.247+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:49.333 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:49.330+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:50.241 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:50.239+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:50.341 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:50.339+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:51.263 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:51.261+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:51.305 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:51.303+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:52.287 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:52.285+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:52.294 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:52.292+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:53.320 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:53.318+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:53.338 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:53.336+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:54.316 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:54.314+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:54.337 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:54.335+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:55.297 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:55.295+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:55.307 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:55.305+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:56.295 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:56.293+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:56.315 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:56.313+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:57.269 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:57.267+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:57.311 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:57.309+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:58.312 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:58.310+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:58.348 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:58.346+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:59.312 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:43:59.310+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:43:59.333 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:43:59.330+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:00.270 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:00.267+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:00.319 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:00.317+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:01.308 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:01.306+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:01.313 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:01.311+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:02.332 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:02.329+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:02.339 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:02.337+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:03.307 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:03.304+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:03.361 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:03.359+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:04.345 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:04.343+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:04.351 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:04.349+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:05.327 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:05.325+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:05.389 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:05.387+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:06.291 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:06.289+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:06.429 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:06.427+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:07.253 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:07.251+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:07.447 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:07.445+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:08.288 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:08.284+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:08.495 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:08.492+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:09.250 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:09.247+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:09.493 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:09.490+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:10.237 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:10.234+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:10.467 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:10.464+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:11.206 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:11.202+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:11.503 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:11.500+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:12.209 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:12.206+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:12.485 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:12.482+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:13.209 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:13.206+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:13.470 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:13.467+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:14.169 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:14.166+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:14.426 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:14.421+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:15.182 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:15.179+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:15.452 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:15.449+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:16.181 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:16.178+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:16.464 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:16.461+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:17.153 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:17.150+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:17.436 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:17.433+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:18.161 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:18.158+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:18.411 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:18.408+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:19.183 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:19.180+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:19.436 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:19.433+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:20.218 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:20.214+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:20.408 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:20.405+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:21.228 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:21.224+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:21.386 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:21.383+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:22.243 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:22.240+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:22.418 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:22.413+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:23.259 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:23.256+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:23.386 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:23.383+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:24.228 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:24.225+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:24.406 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:24.403+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:25.244 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:25.241+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:25.438 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:25.432+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:26.264 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:26.261+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:26.425 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:26.422+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:27.276 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:27.272+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:27.407 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:27.404+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:28.313 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:28.310+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:28.446 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:28.443+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:29.265 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:29.262+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:29.446 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:29.442+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:30.299 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:30.296+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:30.443 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:30.440+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:31.325 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:31.322+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:31.477 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:31.473+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:32.361 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:32.357+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:32.495 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:32.492+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:33.339 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:33.336+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:33.545 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:33.542+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:34.343 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:34.340+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:34.544 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:34.541+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:35.391 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:35.388+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:35.505 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:35.502+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:36.409 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:36.406+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:36.517 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:36.514+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:37.408 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:37.405+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:37.532 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:37.528+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:38.425 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:38.422+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:38.552 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:38.550+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:39.446 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:39.443+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:39.514 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:39.511+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:40.459 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:40.456+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:40.513 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:40.510+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:41.446 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:41.443+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:41.505 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:41.502+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:42.438 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:42.436+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:42.514 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:42.512+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:43.477 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:43.475+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:43.523 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:43.521+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:44.453 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:44.451+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:44.476 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:44.474+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:45.455 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:45.453+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:45.508 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:45.506+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:46.473 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:46.471+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:46.476 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:46.474+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:47.446 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:47.444+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:47.482 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:47.480+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:48.448 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:48.446+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:48.522 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:48.520+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:49.456 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:49.454+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:49.530 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:49.528+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:50.476 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:50.474+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:50.510 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:50.508+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:51.433 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:51.431+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:51.499 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:51.497+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:52.459 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:52.457+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:52.472 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:52.470+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:53.458 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:53.456+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:53.506 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:53.504+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:54.499 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:54.497+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:54.525 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:54.523+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:55.521 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:55.519+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:55.527 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:55.525+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:56.546 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:56.544+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:56.567 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:56.565+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:57.526 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:57.523+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:57.580 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:57.578+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:58.498 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:58.496+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:58.581 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:58.579+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:59.544 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:44:59.542+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:44:59.580 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:44:59.578+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:00.541 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:00.536+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:00.582 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:00.579+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:01.504 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:01.501+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:01.551 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:01.548+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:02.482 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:02.479+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:02.508 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:02.505+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:03.488 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:03.485+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:03.493 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:03.490+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:04.512 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:04.509+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:04.540 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:04.537+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:05.522 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:05.518+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:05.562 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:05.559+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:06.522 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:06.519+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:06.584 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:06.581+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:07.545 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:07.542+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:07.555 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:07.552+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:08.567 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:08.564+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:08.584 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:08.581+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:09.524 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:09.521+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:09.562 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:09.559+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:10.530 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:10.526+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:10.600 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:10.597+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:11.514 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:11.511+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:11.552 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:11.548+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:12.482 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:12.478+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:12.580 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:12.577+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:13.451 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:13.448+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:13.554 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:13.551+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:14.412 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:14.409+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:14.563 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:14.560+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:15.447 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:15.444+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:15.540 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:15.537+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:16.430 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:16.427+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:16.533 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:16.529+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:17.425 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:17.421+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:17.556 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:17.553+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:18.440 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:18.437+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:18.524 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:18.521+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:19.480 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:19.477+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:19.487 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:19.484+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:20.440 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:20.437+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:20.484 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:20.481+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:21.435 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:21.432+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:21.444 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:21.441+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:22.410 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:22.407+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:22.426 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:22.423+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:23.450 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:23.446+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:23.460 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:23.457+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:24.420 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:24.417+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:24.431 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:24.428+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:25.449 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:25.445+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:25.467 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:25.464+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:26.418 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:26.415+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:26.496 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:26.493+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:27.420 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:27.417+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:27.464 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:27.461+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:28.405 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:28.402+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:28.448 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:28.445+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:29.391 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:29.388+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:29.410 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:29.407+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:30.403 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:30.400+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:30.426 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:30.423+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:31.400 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:31.397+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:31.472 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:31.469+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:32.358 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:32.354+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:32.446 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:32.443+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:33.323 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:33.320+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:33.426 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:33.424+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:34.324 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:34.322+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:34.414 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:34.412+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:35.368 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:35.366+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:35.452 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:35.450+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:36.364 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:36.362+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:36.468 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:36.466+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:37.323 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:37.321+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:37.514 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:37.512+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:38.323 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:38.321+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:38.534 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:38.532+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:39.344 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:39.342+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:39.504 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:39.502+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:40.314 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:40.312+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:40.514 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:40.512+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:41.267 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:41.265+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:41.559 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:41.557+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:42.305 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:42.303+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:42.525 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:42.523+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:43.260 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:43.258+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:43.496 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:43.494+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:44.292 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:44.290+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:44.480 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:44.478+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:45.301 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:45.299+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:45.501 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:45.499+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:46.256 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:46.254+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:46.506 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:46.504+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:47.304 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:47.302+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:47.537 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:47.535+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:48.354 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:48.352+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:48.525 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:48.523+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:49.376 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:49.374+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:49.518 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:49.516+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:50.356 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:50.354+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:50.552 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:50.550+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:51.398 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:51.395+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:51.600 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:51.597+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:52.356 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:52.353+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:52.560 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:52.557+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:53.404 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:53.401+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:53.529 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:53.526+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:54.442 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:54.439+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:54.524 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:54.521+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:55.454 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:55.451+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:55.572 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:55.569+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:56.413 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:56.410+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:56.550 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:56.547+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:57.411 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:57.408+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:57.543 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:57.540+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:58.386 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:58.383+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:58.510 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:58.507+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:59.355 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:45:59.352+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:45:59.516 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:45:59.513+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:00.382 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:00.379+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:00.538 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:00.535+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:01.406 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:01.403+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:01.584 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:01.581+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:02.420 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:02.417+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:02.595 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:02.592+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:03.440 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:03.437+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:03.569 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:03.566+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:04.430 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:04.427+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:04.549 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:04.546+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:05.472 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:05.469+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:05.536 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:05.533+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:06.438 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:06.435+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:06.527 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:06.523+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:07.437 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:07.434+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:07.512 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:07.509+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:08.476 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:08.473+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:08.486 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:08.483+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:09.491 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:09.488+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:09.520 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:09.517+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:10.490 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:10.487+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:10.520 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:10.517+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:11.456 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:11.453+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:11.568 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:11.565+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:12.476 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:12.473+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:12.569 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:12.566+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:13.519 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:13.516+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:13.596 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:13.593+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:14.509 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:14.504+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:14.594 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:14.590+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:15.475 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:15.471+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:15.633 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:15.630+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:16.455 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:16.451+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:16.677 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:16.673+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:17.465 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:17.462+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:17.702 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:17.699+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:18.505 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:18.502+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:18.678 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:18.675+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:19.500 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:19.497+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:19.688 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:19.684+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:20.538 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:20.535+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:20.712 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:20.709+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:21.563 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:21.560+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:21.760 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:21.757+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:22.550 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:22.547+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:22.747 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:22.745+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:23.560 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:23.557+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:23.717 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:23.714+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:24.534 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:24.531+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:24.722 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:24.719+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:25.550 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:25.548+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:25.675 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:25.673+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:26.569 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:26.567+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:26.626 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:26.624+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:27.571 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:27.569+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:27.580 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:27.578+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:28.575 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:28.573+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:28.593 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:28.591+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:29.553 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:29.551+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:29.587 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:29.585+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:30.567 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:30.565+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:30.591 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:30.589+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:31.574 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:31.571+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:31.587 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:31.585+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:32.563 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:32.561+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:32.605 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:32.603+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:33.606 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:33.604+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:33.613 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:33.611+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:34.581 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:34.579+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:34.651 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:34.649+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:35.541 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:35.539+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:35.638 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:35.636+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:36.529 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:36.527+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:36.592 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:36.590+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:37.509 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:37.507+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:37.547 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:37.544+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:38.525 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:38.523+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:38.550 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:38.548+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:39.525 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:39.523+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:39.541 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:39.539+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:40.542 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:40.540+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:40.564 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:40.562+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:41.545 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:41.543+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:41.606 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:41.604+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:42.589 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:42.587+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:42.591 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:42.589+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:43.547 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:43.544+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:43.574 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:43.572+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:44.543 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:44.540+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:44.567 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:44.564+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:45.501 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:45.498+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:45.567 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:45.564+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:46.501 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:46.497+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:46.549 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:46.545+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:47.534 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:47.531+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:47.571 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:47.568+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:48.583 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:48.580+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:48.611 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:48.608+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:49.601 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:49.598+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:49.647 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:49.644+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:50.589 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:50.584+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:50.603 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:50.600+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:51.557 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:51.554+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:51.653 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:51.650+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:52.547 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:52.544+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:52.637 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:52.634+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:53.547 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:53.544+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:53.599 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:53.596+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:54.545 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:54.542+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:54.589 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:54.586+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:55.557 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:55.554+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:55.635 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:55.632+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:56.594 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:56.590+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:56.629 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:56.626+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:57.605 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:57.600+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:57.651 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:57.648+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:58.563 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:58.560+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:58.697 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:58.694+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:59.519 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:46:59.516+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:46:59.695 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:46:59.692+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:00.544 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:00.540+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:00.739 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:00.736+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:01.583 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:01.580+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:01.737 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:01.734+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:02.623 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:02.620+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:02.724 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:02.721+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:03.663 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:03.660+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:03.720 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:03.717+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:04.677 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:04.673+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:04.709 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:04.706+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:05.701 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:05.698+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:05.709 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:05.706+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:06.697 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:06.694+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:06.757 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:06.754+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:07.676 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:07.673+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:07.776 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:07.773+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:08.676 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:08.673+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:08.820 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:08.817+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:09.717 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:09.714+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:09.793 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:09.790+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:10.698 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:10.695+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:10.828 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:10.824+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:11.658 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:11.655+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:11.809 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:11.806+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:12.704 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:12.701+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:12.785 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:12.782+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:13.658 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:13.655+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:13.786 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:13.783+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:14.678 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:14.675+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:14.814 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:14.811+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:15.681 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:15.678+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:15.825 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:15.822+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:16.711 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:16.709+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:16.867 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:16.865+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:17.670 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:17.668+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:17.831 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:17.829+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:18.623 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:18.621+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:18.810 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:18.808+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:19.617 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:19.615+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:19.857 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:19.855+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:20.656 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:20.654+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:20.880 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:20.878+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:21.628 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:21.626+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:21.842 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:21.840+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:22.638 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:22.635+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:22.801 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:22.799+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:23.607 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:23.605+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:23.756 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:23.754+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:24.586 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:24.584+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:24.708 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:24.706+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:25.544 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:25.541+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:25.721 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:25.719+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:26.542 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:26.540+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:26.700 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:26.698+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:27.573 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:27.571+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:27.717 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:27.715+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:28.602 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:28.600+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:28.686 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:28.683+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:29.558 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:29.556+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:29.690 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:29.688+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:30.572 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:30.570+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:30.650 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:30.648+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:31.581 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:31.579+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:31.622 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:31.620+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:32.565 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:32.563+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:32.660 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:32.658+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:33.597 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:33.595+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:33.643 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:33.641+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:34.640 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:34.637+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:34.688 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:34.685+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:35.631 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:35.628+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:35.684 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:35.681+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:36.655 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:36.652+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:36.709 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:36.706+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:37.657 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:37.653+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:37.701 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:37.698+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:38.632 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:38.629+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:38.666 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:38.663+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:39.621 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:39.618+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:39.665 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:39.662+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:40.640 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:40.637+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:40.659 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:40.656+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:41.648 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:41.645+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:41.667 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:41.664+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:42.614 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:42.610+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:42.698 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:42.695+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:43.649 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:43.646+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:43.737 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:43.734+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:44.622 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:44.619+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:44.734 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:44.731+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:45.591 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:45.588+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:45.751 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:45.748+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:46.560 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:46.557+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:46.780 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:46.777+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:47.548 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:47.545+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:47.783 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:47.779+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:48.514 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:48.510+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:48.770 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:48.767+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:49.552 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:49.548+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:49.754 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:49.750+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:50.564 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:50.561+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:50.710 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:50.707+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:51.540 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:51.537+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:51.743 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:51.740+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:52.540 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:52.537+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:52.734 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:52.731+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:53.544 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:53.540+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:53.745 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:53.742+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:54.538 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:54.535+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:54.724 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:54.721+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:55.581 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:55.578+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:55.719 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:55.715+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:56.582 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:56.579+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:56.714 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:56.710+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:57.582 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:57.579+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:57.712 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:57.709+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:58.609 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:58.605+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:58.669 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:58.666+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:59.634 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:47:59.631+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:47:59.679 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:47:59.676+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:00.645 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:00.642+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:00.708 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:00.705+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:01.664 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:01.661+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:01.676 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:01.674+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:02.636 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:02.633+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:02.645 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:02.642+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:03.663 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:03.659+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:03.678 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:03.675+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:04.641 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:04.638+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:04.701 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:04.698+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:05.659 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:05.656+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:05.726 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:05.723+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:06.673 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:06.669+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:06.717 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:06.714+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:07.694 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:07.691+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:07.705 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:07.702+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:08.676 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:08.674+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:08.689 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:08.687+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:09.680 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:09.678+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:09.693 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:09.691+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:10.694 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:10.691+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:10.698 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:10.696+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:11.688 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:11.686+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:11.696 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:11.694+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:12.655 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:12.653+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:12.741 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:12.739+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:13.645 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:13.643+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:13.751 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:13.749+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:14.620 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:14.618+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:14.718 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:14.716+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:15.629 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:15.627+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:15.713 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:15.711+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:16.586 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:16.583+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:16.751 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:16.749+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:17.586 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:17.583+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:17.789 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:17.787+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:18.585 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:18.583+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:18.781 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:18.779+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:19.562 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:19.560+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:19.797 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:19.795+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:20.574 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:20.572+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:20.749 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:20.747+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:21.554 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:21.552+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:21.798 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:21.796+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:22.573 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:22.571+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:22.788 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:22.786+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:23.606 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:23.604+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:23.768 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:23.766+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:24.589 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:24.586+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:24.763 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:24.761+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:25.591 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:25.589+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:25.784 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:25.782+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:26.624 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:26.621+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:26.736 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:26.733+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:27.581 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:27.578+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:27.767 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:27.763+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:28.580 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:28.577+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:28.733 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:28.730+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:29.561 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:29.558+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:29.782 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:29.779+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:30.538 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:30.535+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:30.797 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:30.794+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:31.493 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:31.490+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:31.832 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:31.829+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:32.460 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:32.456+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:32.876 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:32.873+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:33.510 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:33.507+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:33.839 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:33.836+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:34.526 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:34.523+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:34.868 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:34.865+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:35.493 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:35.490+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:35.871 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:35.868+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:36.527 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:36.524+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:36.853 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:36.850+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:37.533 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:37.530+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:37.841 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:37.838+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:38.573 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:38.570+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:38.805 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:38.802+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:39.584 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:39.581+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:39.819 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:39.816+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:40.634 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:40.630+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:40.793 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:40.790+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:41.645 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:41.642+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:41.785 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:41.782+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:42.639 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:42.636+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:42.739 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:42.736+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:43.629 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:43.625+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:43.719 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:43.716+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:44.674 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:44.671+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:44.685 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:44.682+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:45.647 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:45.644+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:45.697 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:45.694+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:46.610 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:46.607+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:46.677 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:46.674+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:47.575 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:47.572+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:47.725 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:47.722+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:48.557 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:48.554+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:48.774 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:48.771+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:49.538 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:49.535+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:49.750 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:49.747+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:50.581 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:50.577+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:50.723 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:50.720+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:51.533 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:51.530+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:51.748 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:51.745+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:52.491 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:52.488+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:52.743 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:52.740+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:53.505 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:53.502+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:53.746 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:53.743+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:54.461 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:54.458+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:54.782 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:54.779+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:55.415 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:55.412+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:55.765 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:55.762+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:56.417 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:56.414+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:56.789 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:56.786+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:57.459 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:57.456+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:57.838 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:57.835+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:58.489 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:58.486+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:58.877 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:58.874+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:59.518 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:48:59.516+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:48:59.883 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:48:59.881+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:00.526 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:00.523+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:00.932 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:00.930+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:01.554 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:01.552+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:01.920 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:01.917+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:02.508 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:02.505+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:02.906 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:02.903+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:03.505 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:03.503+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:03.948 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:03.946+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:04.499 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:04.497+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:04.909 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:04.907+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:05.490 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:05.488+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:05.885 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:05.883+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:06.472 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:06.470+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:06.850 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:06.848+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:07.431 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:07.429+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:07.888 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:07.886+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:08.432 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:08.430+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:08.930 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:08.928+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:09.477 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:09.475+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:09.969 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:09.967+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:10.528 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:10.526+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:11.019 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:11.017+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:11.508 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:11.506+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:11.972 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:11.970+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:12.466 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:12.464+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:13.005 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:13.002+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:13.442 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:13.440+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:13.984 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:13.982+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:14.404 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:14.402+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:14.939 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:14.937+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:15.378 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:15.375+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:15.946 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:15.944+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:16.395 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:16.393+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:16.994 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:16.992+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:17.382 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:17.380+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:18.006 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:18.003+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:18.422 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:18.419+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:19.006 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:19.002+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:19.376 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:19.373+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:20.037 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:20.034+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:20.417 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:20.414+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:20.998 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:20.995+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:21.430 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:21.427+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:22.048 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:22.045+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:22.437 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:22.433+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:23.088 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:23.085+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:23.406 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:23.403+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:24.070 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:24.067+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:24.432 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:24.429+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:25.102 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:25.098+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:25.454 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:25.451+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:26.059 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:26.056+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:26.493 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:26.490+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:27.102 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:27.099+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:27.465 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:27.462+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:28.137 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:28.134+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:28.452 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:28.448+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:29.163 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:29.160+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:29.500 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:29.497+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:30.129 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:30.126+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:30.548 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:30.545+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:31.082 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:31.079+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:31.564 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:31.560+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:32.079 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:32.076+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:32.611 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:32.608+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:33.049 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:33.046+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:33.595 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:33.592+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:34.098 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:34.095+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:34.584 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:34.581+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:35.130 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:35.127+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:35.622 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:35.619+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:36.087 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:36.084+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:36.661 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:36.658+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:37.090 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:37.087+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:37.686 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:37.683+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:38.118 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:38.115+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:38.709 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:38.705+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:39.122 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:39.119+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:39.684 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:39.681+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:40.080 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:40.077+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:40.638 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:40.635+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:41.076 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:41.073+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:41.672 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:41.669+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:42.051 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:42.048+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:42.706 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:42.703+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:43.027 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:43.024+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:43.711 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:43.708+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:44.008 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:44.005+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:44.734 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:44.731+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:45.042 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:45.039+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:45.718 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:45.715+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:45.996 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:45.993+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:46.753 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:46.750+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:47.036 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:47.033+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:47.780 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:47.777+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:48.072 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:48.069+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:48.765 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:48.761+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:49.048 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:49.045+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:49.806 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:49.803+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:50.006 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:50.003+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:50.835 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:50.832+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:50.993 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:50.991+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:51.867 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:51.865+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:51.971 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:51.969+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:52.821 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:52.819+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:52.928 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:52.926+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:53.817 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:53.815+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:53.926 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:53.924+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:54.771 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:54.769+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:54.946 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:54.944+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:55.749 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:55.747+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:55.967 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:55.965+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:56.777 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:56.775+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:57.007 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:57.005+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:57.735 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:57.733+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:57.991 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:57.989+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:58.744 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:58.742+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:59.013 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:49:59.011+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:49:59.717 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:49:59.715+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:00.033 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:00.031+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:00.677 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:00.675+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:01.025 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:01.023+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:01.683 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:01.681+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:01.995 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:01.993+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:02.686 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:02.684+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:02.967 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:02.965+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:03.664 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:03.662+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:03.939 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:03.937+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:04.697 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:04.694+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:04.936 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:04.934+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:05.692 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:05.690+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:05.974 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:05.972+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:06.647 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:06.645+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:06.991 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:06.989+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:07.620 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:07.618+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:08.030 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:08.028+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:08.643 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:08.641+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:09.070 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:09.067+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:09.647 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:09.644+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:10.027 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:10.024+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:10.668 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:10.665+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:11.014 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:11.011+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:11.682 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:11.679+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:12.009 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:12.006+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:12.682 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:12.679+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:12.963 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:12.960+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:13.726 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:13.723+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:13.971 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:13.968+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:14.706 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:14.703+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:14.993 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:14.989+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:15.670 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:15.667+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:16.003 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:16.000+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:16.650 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:16.647+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:17.039 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:17.036+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:17.681 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:17.678+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:17.998 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:17.995+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:18.667 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:18.664+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:19.011 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:19.008+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:19.703 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:19.700+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:20.015 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:20.012+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:20.675 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:20.672+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:20.988 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:20.984+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:21.631 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:21.628+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:21.999 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:21.996+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:22.600 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:22.597+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:22.955 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:22.952+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:23.637 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:23.634+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:23.942 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:23.939+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:24.655 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:24.652+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:24.955 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:24.952+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:25.640 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:25.637+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:25.945 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:25.941+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:26.673 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:26.670+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:26.957 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:26.954+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:27.632 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:27.629+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:27.941 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:27.938+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:28.641 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:28.638+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:28.984 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:28.981+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:29.675 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:29.672+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:30.013 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:30.010+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:30.726 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:30.723+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:31.064 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:31.061+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:31.692 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:31.689+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:32.054 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:32.051+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:32.726 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:32.723+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:33.100 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:33.097+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:33.750 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:33.747+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:34.083 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:34.080+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:34.779 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:34.776+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:35.063 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:35.060+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:35.762 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:35.758+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:36.041 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:36.035+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:36.779 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:36.776+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:36.995 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:36.992+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:37.801 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:37.798+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:37.948 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:37.945+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:38.767 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:38.764+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:38.996 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:38.993+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:39.757 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:39.754+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:39.989 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:39.986+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:40.719 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:40.716+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:41.021 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:41.018+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:41.693 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:41.690+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:42.019 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:42.016+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:42.685 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:42.683+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:43.011 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:43.008+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:43.636 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:43.634+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:43.993 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:43.991+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:44.681 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:44.679+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:44.981 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:44.979+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:45.655 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:45.653+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:46.020 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:46.018+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:46.697 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:46.695+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:46.985 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:46.983+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:47.685 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:47.683+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:47.965 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:47.963+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:48.724 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:48.722+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:49.005 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:49.003+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:49.713 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:49.711+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:49.981 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:49.979+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:50.683 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:50.680+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:50.935 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:50.933+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:51.701 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:51.699+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:51.951 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:51.949+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:52.692 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:52.690+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:52.910 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:52.908+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:53.677 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:53.675+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:53.928 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:53.926+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:54.643 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:54.641+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:54.925 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:54.923+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:55.685 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:55.683+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:55.949 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:55.947+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:56.681 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:56.679+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:56.969 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:56.967+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:57.645 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:57.643+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:57.988 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:57.985+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:58.621 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:58.619+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:58.958 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:58.956+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:59.619 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:50:59.617+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:50:59.979 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:50:59.977+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:00.584 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:00.582+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:00.985 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:00.983+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:01.582 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:01.579+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:02.015 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:02.012+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:02.543 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:02.540+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:03.053 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:03.050+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:03.565 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:03.562+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:04.059 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:04.056+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:04.610 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:04.607+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:05.023 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:05.020+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:05.592 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:05.589+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:06.060 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:06.057+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:06.590 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:06.587+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:07.107 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:07.104+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:07.558 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:07.555+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:08.155 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:08.152+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:08.589 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:08.586+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:09.137 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:09.134+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:09.609 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:09.606+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:10.171 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:10.168+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:10.591 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:10.588+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:11.169 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:11.166+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:11.596 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:11.592+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:12.144 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:12.141+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:12.633 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:12.630+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:13.114 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:13.111+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:13.625 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:13.622+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:14.085 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:14.082+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:14.626 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:14.623+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:15.089 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:15.086+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:15.594 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:15.591+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:16.089 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:16.086+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:16.630 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:16.626+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:17.071 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:17.068+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:17.666 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:17.663+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:18.048 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:18.045+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:18.699 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:18.696+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:19.038 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:19.035+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:19.740 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:19.737+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:20.088 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:20.085+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:20.737 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:20.734+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:21.073 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:21.070+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:21.777 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:21.774+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:22.046 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:22.043+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:22.768 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:22.765+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:23.047 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:23.044+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:23.755 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:23.752+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:24.067 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:24.063+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:24.716 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:24.713+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:25.059 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:25.056+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:25.753 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:25.750+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:26.107 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:26.104+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:26.727 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:26.724+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:27.098 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:27.095+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:27.699 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:27.696+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:28.120 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:28.116+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:28.715 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:28.712+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:29.169 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:29.166+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:29.678 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:29.675+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:30.161 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:30.158+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:30.672 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:30.669+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:31.195 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:31.192+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:31.696 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:31.693+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:32.148 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:32.145+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:32.710 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:32.707+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:33.113 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:33.110+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:33.743 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:33.739+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:34.106 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:34.103+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:34.739 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:34.736+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:35.066 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:35.063+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:35.721 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:35.719+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:36.025 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:36.023+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:36.694 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:36.692+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:36.985 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:36.983+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:37.651 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:37.649+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:37.995 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:37.993+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:38.647 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:38.645+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:38.991 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:38.989+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:39.649 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:39.647+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:40.012 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:40.010+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:40.657 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:40.654+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:41.016 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:41.014+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:41.647 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:41.645+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:41.991 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:41.989+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:42.630 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:42.628+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:43.017 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:43.015+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:43.620 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:43.618+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:44.048 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:44.046+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:44.670 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:44.668+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:45.047 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:45.045+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:45.633 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:45.631+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:46.003 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:46.001+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:46.585 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:46.583+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:46.974 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:46.972+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:47.605 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:47.603+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:47.951 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:47.949+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:48.649 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:48.647+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:48.967 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:48.965+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:49.668 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:49.666+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:49.987 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:49.985+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:50.678 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:50.676+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:50.967 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:50.965+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:51.722 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:51.720+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:52.016 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:52.014+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:52.702 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:52.700+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:52.977 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:52.975+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:53.665 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:53.662+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:53.958 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:53.955+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:54.663 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:54.660+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:54.947 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:54.944+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:55.621 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:55.618+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:55.907 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:55.904+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:56.573 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:56.570+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:56.872 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:56.868+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:57.535 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:57.532+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:57.903 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:57.900+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:58.571 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:58.568+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:58.949 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:58.946+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:59.595 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:51:59.592+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:51:59.986 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:51:59.983+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:00.561 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:00.558+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:00.959 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:00.956+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:01.560 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:01.557+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:01.991 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:01.988+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:02.559 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:02.556+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:02.955 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:02.952+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:03.533 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:03.529+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:03.945 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:03.942+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:04.574 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:04.571+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:04.911 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:04.908+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:05.538 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:05.535+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:05.909 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:05.906+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:06.493 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:06.490+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:06.927 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:06.924+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:07.497 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:07.494+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:07.972 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:07.969+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:08.511 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:08.508+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:08.953 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:08.950+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:09.486 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:09.483+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:09.963 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:09.960+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:10.493 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:10.490+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:10.987 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:10.984+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:11.453 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:11.450+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:11.951 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:11.948+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:12.457 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:12.454+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:12.933 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:12.930+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:13.439 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:13.436+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:13.960 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:13.957+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:14.464 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:14.461+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:14.995 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:14.992+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:15.446 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:15.443+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:15.948 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:15.945+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:16.449 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:16.446+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:16.963 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:16.960+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:17.425 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:17.422+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:17.961 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:17.958+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:18.396 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:18.393+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:18.939 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:18.936+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:19.434 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:19.431+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:19.962 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:19.959+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:20.402 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:20.399+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:20.943 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:20.939+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:21.395 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:21.392+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:21.934 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:21.931+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:22.446 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:22.443+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:22.981 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:22.977+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:23.431 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:23.428+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:23.980 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:23.977+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:24.418 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:24.415+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:24.977 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:24.974+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:25.457 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:25.454+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:26.019 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:26.016+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:26.450 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:26.447+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:27.043 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:27.041+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:27.479 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:27.477+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:28.024 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:28.022+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:28.513 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:28.511+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:29.060 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:29.058+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:29.502 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:29.500+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:30.014 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:30.012+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:30.488 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:30.486+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:31.011 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:31.009+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:31.446 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:31.444+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:32.052 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:32.050+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:32.446 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:32.444+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:33.068 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:33.066+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:33.426 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:33.424+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:34.078 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:34.076+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:34.431 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:34.429+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:35.031 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:35.029+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:35.465 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:35.463+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:36.072 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:36.070+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:36.483 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:36.481+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:37.047 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:37.045+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:37.502 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:37.500+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:38.081 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:38.079+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:38.490 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:38.488+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:39.120 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:39.118+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:39.506 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:39.504+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:40.148 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:40.146+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:40.525 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:40.523+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:41.137 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:41.136+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:41.548 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:41.546+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:42.126 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:42.124+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:42.536 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:42.534+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:43.112 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:43.110+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:43.512 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:43.510+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:44.119 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:44.117+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:44.528 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:44.526+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:45.162 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:45.159+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:45.487 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:45.484+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:46.118 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:46.115+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:46.466 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:46.463+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:47.146 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:47.143+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:47.473 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:47.470+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:48.111 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:48.108+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:48.430 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:48.427+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:49.155 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:49.152+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:49.469 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:49.466+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:50.176 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:50.173+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:50.514 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:50.511+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:51.180 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:51.177+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:51.495 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:51.492+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:52.220 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:52.217+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:52.510 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:52.505+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:53.266 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:53.263+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:53.512 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:53.509+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:54.231 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:54.228+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:54.527 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:54.524+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:55.279 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:55.276+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:55.546 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:55.543+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:56.318 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:56.315+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:56.518 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:56.515+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:57.339 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:57.336+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:57.508 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:57.505+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:58.350 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:58.347+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:58.535 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:58.532+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:59.376 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:52:59.373+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:52:59.514 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:52:59.511+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:00.417 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:00.413+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:00.480 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:00.477+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:01.463 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:01.460+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:01.478 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:01.475+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:02.451 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:02.448+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:02.505 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:02.502+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:03.445 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:03.442+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:03.528 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:03.525+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:04.404 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:04.401+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:04.555 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:04.552+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:05.424 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:05.421+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:05.553 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:05.550+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:06.395 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:06.392+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:06.569 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:06.566+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:07.401 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:07.398+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:07.580 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:07.577+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:08.378 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:08.375+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:08.610 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:08.607+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:09.407 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:09.404+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:09.596 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:09.593+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:10.360 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:10.357+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:10.564 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:10.561+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:11.363 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:11.360+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:11.552 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:11.549+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:12.349 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:12.346+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:12.510 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:12.507+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:13.381 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:13.377+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:13.479 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:13.476+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:14.395 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:14.392+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:14.446 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:14.443+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:15.377 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:15.373+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:15.439 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:15.436+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:16.392 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:16.389+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:16.403 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:16.400+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:17.423 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:17.420+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:17.444 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:17.441+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:18.395 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:18.393+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:18.410 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:18.407+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:19.385 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:19.383+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:19.450 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:19.448+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:20.342 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:20.340+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:20.434 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:20.432+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:21.309 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:21.307+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:21.414 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:21.412+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:22.344 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:22.342+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:22.385 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:22.383+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:23.362 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:23.360+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:23.367 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:23.365+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:24.398 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:24.396+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:24.411 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:24.409+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:25.386 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:25.384+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:25.444 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:25.442+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:26.406 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:26.404+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:26.432 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:26.430+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:27.408 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:27.406+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:27.427 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:27.425+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:28.380 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:28.378+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:28.389 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:28.387+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:29.384 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:29.382+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:29.408 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:29.406+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:30.342 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:30.340+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:30.394 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:30.392+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:31.372 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:31.370+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:31.409 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:31.408+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:32.340 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:32.338+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:32.383 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:32.381+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:33.351 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:33.349+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:33.434 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:33.432+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:34.306 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:34.304+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:34.406 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:34.403+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:35.281 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:35.279+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:35.384 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:35.382+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:36.251 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:36.249+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:36.416 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:36.414+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:37.242 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:37.239+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:37.401 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:37.398+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:38.220 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:38.216+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:38.390 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:38.386+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:39.196 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:39.193+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:39.414 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:39.411+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:40.200 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:40.197+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:40.421 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:40.418+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:41.230 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:41.227+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:41.405 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:41.403+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:42.216 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:42.213+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:42.453 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:42.450+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:43.257 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:43.255+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:43.461 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:43.458+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:44.275 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:44.272+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:44.442 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:44.439+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:45.293 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:45.290+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:45.483 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:45.480+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:46.338 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:46.335+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:46.481 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:46.478+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:47.342 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:47.339+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:47.482 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:47.479+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:48.386 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:48.383+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:48.439 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:48.436+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:49.344 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:49.341+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:49.435 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:49.432+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:50.314 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:50.311+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:50.461 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:50.459+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:51.360 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:51.357+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:51.426 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:51.423+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:52.322 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:52.319+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:52.400 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:52.397+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:53.294 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:53.291+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:53.385 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:53.382+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:54.288 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:54.285+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:54.377 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:54.374+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:55.298 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:55.295+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:55.332 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:55.329+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:56.322 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:56.319+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:56.333 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:56.330+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:57.298 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:57.295+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:57.321 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:57.318+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:58.267 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:58.264+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:58.354 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:58.351+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:59.306 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:53:59.303+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:53:59.385 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:53:59.382+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:00.303 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:00.299+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:00.356 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:00.353+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:01.345 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:01.343+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:01.386 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:01.383+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:02.304 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:02.301+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:02.432 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:02.429+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:03.260 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:03.257+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:03.434 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:03.430+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:04.304 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:04.301+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:04.460 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:04.457+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:05.281 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:05.278+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:05.450 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:05.447+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:06.253 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:06.250+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:06.452 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:06.449+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:07.240 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:07.237+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:07.454 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:07.451+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:08.247 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:08.244+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:08.453 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:08.451+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:09.255 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:09.252+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:09.493 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:09.490+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:10.278 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:10.275+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:10.481 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:10.478+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:11.266 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:11.264+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:11.512 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:11.510+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:12.224 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:12.222+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:12.480 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:12.478+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:13.212 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:13.210+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:13.513 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:13.511+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:14.256 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:14.254+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:14.543 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:14.541+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:15.217 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:15.215+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:15.527 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:15.525+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:16.189 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:16.187+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:16.518 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:16.516+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:17.172 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:17.170+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:17.490 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:17.488+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:18.193 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:18.191+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:18.477 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:18.475+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:19.204 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:19.202+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:19.488 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:19.486+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:20.185 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:20.183+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:20.473 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:20.471+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:21.227 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:21.225+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:21.474 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:21.471+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:22.218 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:22.216+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:22.513 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:22.511+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:23.222 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:23.220+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:23.511 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:23.508+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:24.237 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:24.235+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:24.495 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:24.493+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:25.274 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:25.272+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:25.522 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:25.520+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:26.297 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:26.295+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:26.544 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:26.542+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:27.336 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:27.334+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:27.527 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:27.525+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:28.320 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:28.318+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:28.578 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:28.576+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:29.307 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:29.304+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:29.614 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:29.611+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:30.313 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:30.310+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:30.581 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:30.577+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:31.285 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:31.282+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:31.609 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:31.606+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:32.243 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:32.240+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:32.560 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:32.557+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:33.241 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:33.238+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:33.538 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:33.534+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:34.216 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:34.213+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:34.542 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:34.539+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:35.228 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:35.225+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:35.586 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:35.583+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:36.233 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:36.230+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:36.585 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:36.581+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:37.251 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:37.248+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:37.622 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:37.619+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:38.262 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:38.259+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:38.654 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:38.651+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:39.261 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:39.257+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:39.634 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:39.631+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:40.280 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:40.277+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:40.590 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:40.587+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:41.316 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:41.313+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:41.628 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:41.625+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:42.348 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:42.345+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:42.655 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:42.652+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:43.324 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:43.321+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:43.646 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:43.643+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:44.370 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:44.367+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:44.658 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:44.655+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:45.323 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:45.320+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:45.638 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:45.635+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:46.341 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:46.338+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:46.664 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:46.661+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:47.386 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:47.383+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:47.684 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:47.681+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:48.362 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:48.359+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:48.722 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:48.719+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:49.344 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:49.341+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:49.758 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:49.755+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:50.296 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:50.293+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:50.714 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:50.711+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:51.275 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:51.272+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:51.716 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:51.713+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:52.307 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:52.304+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:52.695 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:52.692+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:53.318 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:53.315+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:53.667 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:53.664+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:54.344 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:54.341+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:54.653 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:54.650+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:55.321 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:55.318+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:55.633 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:55.630+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:56.336 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:56.333+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:56.599 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:56.596+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:57.346 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:57.343+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:57.550 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:57.547+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:58.336 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:58.332+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:58.536 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:58.533+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:59.359 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:54:59.356+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:54:59.555 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:54:59.552+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:00.343 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:00.340+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:00.506 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:00.503+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:01.322 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:01.319+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:01.503 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:01.500+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:02.363 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:02.360+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:02.518 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:02.515+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:03.362 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:03.360+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:03.506 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:03.504+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:04.351 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:04.348+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:04.532 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:04.530+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:05.301 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:05.299+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:05.563 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:05.561+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:06.338 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:06.336+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:06.525 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:06.521+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:07.297 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:07.295+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:07.482 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:07.481+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:08.327 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:08.325+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:08.447 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:08.445+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:09.350 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:09.348+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:09.484 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:09.483+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:10.305 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:10.303+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:10.510 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:10.509+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:11.355 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:11.353+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:11.493 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:11.491+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:12.355 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:12.353+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:12.476 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:12.475+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:13.400 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:13.399+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:13.434 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:13.432+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:14.438 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:14.437+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:14.457 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:14.456+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:15.443 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:15.441+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:15.507 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:15.505+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:16.441 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:16.439+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:16.495 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:16.493+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:17.442 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:17.441+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:17.489 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:17.487+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:18.451 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:18.449+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:18.539 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:18.537+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:19.416 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:19.415+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:19.493 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:19.491+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:20.443 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:20.441+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:20.457 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:20.455+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:21.409 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:21.406+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:21.439 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:21.436+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:22.369 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:22.366+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:22.453 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:22.450+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:23.415 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:23.412+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:23.472 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:23.470+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:24.393 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:24.390+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:24.510 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:24.507+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:25.398 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:25.395+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:25.492 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:25.489+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:26.380 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:26.377+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:26.502 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:26.500+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:27.418 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:27.415+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:27.539 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:27.536+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:28.391 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:28.388+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:28.498 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:28.494+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:29.368 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:29.365+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:29.499 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:29.496+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:30.355 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:30.352+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:30.514 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:30.512+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:31.357 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:31.352+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:31.558 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:31.555+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:32.336 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:32.333+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:32.548 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:32.545+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:33.295 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:33.292+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:33.508 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:33.505+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:34.270 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:34.267+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:34.463 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:34.459+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:35.262 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:35.259+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:35.460 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:35.457+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:36.262 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:36.259+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:36.421 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:36.418+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:37.274 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:37.271+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:37.404 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:37.401+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:38.269 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:38.266+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:38.387 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:38.384+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:39.263 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:39.260+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:39.349 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:39.346+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:40.246 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:40.243+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:40.342 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:40.339+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:41.234 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:41.231+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:41.361 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:41.358+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:42.212 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:42.209+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:42.374 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:42.372+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:43.195 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:43.192+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:43.371 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:43.368+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:44.204 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:44.201+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:44.346 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:44.343+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:45.181 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:45.178+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:45.373 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:45.370+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:46.153 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:46.150+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:46.420 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:46.417+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:47.189 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:47.186+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:47.408 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:47.405+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:48.147 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:48.143+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:48.361 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:48.358+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:49.161 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:49.158+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:49.397 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:49.394+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:50.183 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:50.180+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:50.372 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:50.370+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:51.219 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:51.216+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:51.325 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:51.322+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:52.267 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:52.264+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:52.276 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:52.273+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:53.288 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:53.285+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:53.321 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:53.318+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:54.269 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:54.266+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:54.364 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:54.362+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:55.304 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:55.302+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:55.329 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:55.328+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:56.294 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:56.292+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:56.366 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:56.363+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:57.268 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:57.266+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:57.377 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:57.375+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:58.280 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:58.278+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:58.391 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:58.390+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:59.306 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:55:59.305+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:55:59.373 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:55:59.371+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:00.330 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:00.328+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:00.341 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:00.340+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:01.306 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:01.304+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:01.373 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:01.371+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:02.326 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:02.324+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:02.356 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:02.354+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:03.328 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:03.326+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:03.349 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:03.347+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:04.339 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:04.337+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:04.385 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:04.383+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:05.323 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:05.321+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:05.351 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:05.349+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:06.329 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:06.327+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:06.381 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:06.380+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:07.322 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:07.320+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:07.388 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:07.386+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:08.331 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:08.329+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:08.370 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:08.368+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:09.327 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:09.325+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:09.381 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:09.380+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:10.344 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:10.342+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:10.385 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:10.383+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:11.324 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:11.322+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:11.341 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:11.339+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:12.318 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:12.316+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:12.353 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:12.351+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:13.296 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:13.292+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:13.380 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:13.377+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:14.304 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:14.301+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:14.351 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:14.347+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:15.290 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:15.287+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:15.397 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:15.395+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:16.242 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:16.239+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:16.382 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:16.379+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:17.208 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:17.205+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:17.397 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:17.394+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:18.228 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:18.226+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:18.354 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:18.351+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:19.194 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:19.191+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:19.360 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:19.357+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:20.175 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:20.173+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:20.378 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:20.375+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:21.220 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:21.217+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:21.425 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:21.422+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:22.250 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:22.247+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:22.473 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:22.470+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:23.245 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:23.243+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:23.450 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:23.447+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:24.235 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:24.233+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:24.470 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:24.468+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:25.214 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:25.211+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:25.459 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:25.456+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:26.212 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:26.209+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:26.482 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:26.479+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:27.262 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:27.259+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:27.468 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:27.465+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:28.245 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:28.242+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:28.450 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:28.447+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:29.208 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:29.205+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:29.416 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:29.413+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:30.221 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:30.219+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:30.381 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:30.378+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:31.192 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:31.189+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:31.374 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:31.371+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:32.232 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:32.229+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:32.406 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:32.403+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:33.269 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:33.266+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:33.434 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:33.431+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:34.310 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:34.307+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:34.441 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:34.438+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:35.299 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:35.296+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:35.436 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:35.433+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:36.346 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:36.343+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:36.444 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:36.441+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:37.302 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:37.299+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:37.396 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:37.393+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:38.272 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:38.269+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:38.394 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:38.391+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:39.287 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:39.284+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:39.371 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:39.368+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:40.266 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:40.263+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:40.390 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:40.388+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:41.277 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:41.274+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:41.429 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:41.427+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:42.326 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:42.323+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:42.389 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:42.386+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:43.338 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:43.335+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:43.341 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:43.338+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:44.291 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:44.288+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:44.320 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:44.317+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:45.268 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:45.265+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:45.296 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:45.294+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:46.233 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:46.230+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:46.330 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:46.327+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:47.273 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:47.271+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:47.329 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:47.327+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:48.298 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:48.296+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:48.301 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:48.300+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:49.298 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:49.296+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:49.327 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:49.325+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:50.322 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:50.320+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:50.348 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:50.346+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:51.300 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:51.298+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:51.311 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:51.309+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:52.270 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:52.268+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:52.343 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:52.342+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:53.244 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:53.242+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:53.372 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:53.370+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:54.242 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:54.240+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:54.364 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:54.362+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:55.283 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:55.281+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:55.324 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:55.322+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:56.323 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:56.321+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:56.348 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:56.346+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:57.348 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:57.346+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:57.391 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:57.389+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:58.388 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:58.386+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:58.389 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:58.387+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:59.388 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:56:59.386+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:56:59.407 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:56:59.406+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:00.383 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:00.381+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:00.454 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:00.452+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:01.342 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:01.339+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:01.459 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:01.458+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:02.354 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:02.352+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:02.420 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:02.418+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:03.397 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:03.395+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:03.426 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:03.424+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:04.435 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:04.432+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:04.456 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:04.453+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:05.473 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:05.470+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:05.473 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:05.470+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:06.443 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:06.441+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:06.481 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:06.478+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:07.400 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:07.397+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:07.510 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:07.507+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:08.368 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:08.365+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:08.547 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:08.544+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:09.404 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:09.401+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:09.503 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:09.501+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:10.408 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:10.405+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:10.522 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:10.519+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:11.409 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:11.406+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:11.517 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:11.513+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:12.418 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:12.415+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:12.544 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:12.541+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:13.431 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:13.428+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:13.500 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:13.497+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:14.394 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:14.391+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:14.520 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:14.517+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:15.430 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:15.427+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:15.478 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:15.475+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:16.404 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:16.401+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:16.447 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:16.445+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:17.361 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:17.358+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:17.474 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:17.471+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:18.348 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:18.345+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:18.435 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:18.432+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:19.397 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:19.394+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:19.456 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:19.453+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:20.354 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:20.351+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:20.414 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:20.411+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:21.348 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:21.345+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:21.436 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:21.433+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:22.391 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:22.388+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:22.466 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:22.463+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:23.373 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:23.370+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:23.446 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:23.443+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:24.343 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:24.340+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:24.432 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:24.429+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:25.310 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:25.307+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:25.455 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:25.453+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:26.277 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:26.274+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:26.436 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:26.433+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:27.288 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:27.285+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:27.399 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:27.396+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:28.300 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:28.297+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:28.430 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:28.427+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:29.346 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:29.343+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:29.475 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:29.473+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:30.381 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:30.378+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:30.516 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:30.513+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:31.420 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:31.417+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:31.480 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:31.477+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:32.405 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:32.402+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:32.500 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:32.497+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:33.409 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:33.406+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:33.538 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:33.535+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:34.445 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:34.442+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:34.570 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:34.568+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:35.397 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:35.394+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:35.555 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:35.552+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:36.437 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:36.434+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:36.578 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:36.576+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:37.430 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:37.428+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:37.622 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:37.619+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:38.391 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:38.389+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:38.596 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:38.595+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:39.407 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:39.405+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:39.588 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:39.587+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:40.455 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:40.453+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:40.571 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:40.569+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:41.431 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:41.429+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:41.580 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:41.578+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:42.385 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:42.383+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:42.560 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:42.559+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:43.434 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:43.432+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:43.583 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:43.581+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:44.440 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:44.438+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:44.535 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:44.533+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:45.457 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:45.455+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:45.528 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:45.526+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:46.465 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:46.463+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:46.518 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:46.517+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:47.495 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:47.493+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:47.518 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:47.516+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:48.459 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:48.457+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:48.539 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:48.537+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:49.475 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:49.473+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:49.556 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:49.554+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:50.432 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:50.430+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:50.580 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:50.578+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:51.392 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:51.390+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:51.600 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:51.598+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:52.403 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:52.401+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:52.558 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:52.556+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:53.447 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:53.445+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:53.575 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:53.573+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:54.436 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:54.435+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:54.597 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:54.595+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:55.433 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:55.432+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:55.635 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:55.633+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:56.470 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:56.467+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:56.639 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:56.636+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:57.491 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:57.488+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:57.650 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:57.648+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:58.443 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:58.440+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:58.679 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:58.676+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:59.438 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:57:59.436+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:57:59.729 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:57:59.726+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:00.476 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:00.473+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:00.681 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:00.678+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:01.446 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:01.443+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:01.663 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:01.660+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:02.488 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:02.486+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:02.618 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:02.616+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:03.441 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:03.438+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:03.631 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:03.628+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:04.459 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:04.456+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:04.595 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:04.592+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:05.489 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:05.486+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:05.551 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:05.548+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:06.508 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:06.505+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:06.553 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:06.550+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:07.549 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:07.546+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:07.590 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:07.587+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:08.561 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:08.558+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:08.577 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:08.574+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:09.550 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:09.548+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:09.627 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:09.625+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:10.530 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:10.526+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:10.647 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:10.644+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:11.578 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:11.575+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:11.622 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:11.619+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:12.560 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:12.557+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:12.657 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:12.654+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:13.553 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:13.550+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:13.648 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:13.645+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:14.555 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:14.552+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:14.635 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:14.632+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:15.553 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:15.550+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:15.649 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:15.646+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:16.573 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:16.570+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:16.683 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:16.680+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:17.567 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:17.564+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:17.694 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:17.691+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:18.569 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:18.566+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:18.700 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:18.698+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:19.556 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:19.553+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:19.726 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:19.723+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:20.514 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:20.511+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:20.753 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:20.750+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:21.504 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:21.501+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:21.779 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:21.776+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:22.507 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:22.505+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:22.774 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:22.771+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:23.529 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:23.526+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:23.821 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:23.818+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:24.496 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:24.493+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:24.787 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:24.784+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:25.483 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:25.480+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:25.793 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:25.790+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:26.503 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:26.500+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:26.813 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:26.810+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:27.530 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:27.527+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:27.784 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:27.782+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:28.565 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:28.563+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:28.751 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:28.748+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:29.542 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:29.540+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:29.757 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:29.755+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:30.495 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:30.493+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:30.780 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:30.779+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:31.469 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:31.467+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:31.736 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:31.734+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:32.424 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:32.422+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:32.730 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:32.729+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:33.387 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:33.385+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:33.705 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:33.703+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:34.373 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:34.371+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:34.657 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:34.655+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:35.421 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:35.419+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:35.689 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:35.687+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:36.455 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:36.453+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:36.660 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:36.659+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:37.443 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:37.442+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:37.698 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:37.697+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:38.466 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:38.464+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:38.693 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:38.691+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:39.490 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:39.488+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:39.705 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:39.703+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:40.467 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:40.465+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:40.723 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:40.721+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:41.425 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:41.423+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:41.769 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:41.767+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:42.386 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:42.384+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:42.791 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:42.789+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:43.352 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:43.350+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:43.765 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:43.763+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:44.351 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:44.349+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:44.740 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:44.738+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:45.317 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:45.315+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:45.785 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:45.783+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:46.319 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:46.317+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:46.803 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:46.801+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:47.277 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:47.274+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:47.775 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:47.772+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:48.304 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:48.302+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:48.806 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:48.804+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:49.321 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:49.318+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:49.840 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:49.837+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:50.347 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:50.344+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:50.883 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:50.880+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:51.357 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:51.354+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:51.895 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:51.892+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:52.394 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:52.391+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:52.864 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:52.862+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:53.433 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:53.430+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:53.910 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:53.907+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:54.431 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:54.429+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:54.948 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:54.945+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:55.449 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:55.447+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:55.953 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:55.950+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:56.484 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:56.482+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:56.995 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:56.992+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:57.535 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:57.532+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:57.981 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:57.978+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:58.530 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:58.527+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:59.031 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:59.028+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:59.509 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:58:59.506+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:58:59.998 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:58:59.996+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:00.523 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:00.520+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:00.962 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:00.959+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:01.524 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:01.521+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:01.986 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:01.983+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:02.513 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:02.510+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:02.943 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:02.939+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:03.538 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:03.534+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:03.984 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:03.981+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:04.521 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:04.518+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:05.000 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:04.998+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:05.541 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:05.538+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:05.971 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:05.968+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:06.537 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:06.534+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:06.937 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:06.934+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:07.542 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:07.539+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:07.987 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:07.984+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:08.533 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:08.530+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:09.021 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:09.018+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:09.506 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:09.504+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:10.020 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:10.018+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:10.517 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:10.514+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:10.990 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:10.987+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:11.516 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:11.514+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:11.980 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:11.977+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:12.545 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:12.542+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:12.938 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:12.936+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:13.569 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:13.566+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:13.930 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:13.927+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:14.605 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:14.602+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:14.940 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:14.937+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:15.625 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:15.622+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:15.912 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:15.909+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:16.606 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:16.603+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:16.898 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:16.896+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:17.612 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:17.610+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:17.914 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:17.911+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:18.587 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:18.584+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:18.878 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:18.876+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:19.542 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:19.539+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:19.845 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:19.843+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:20.541 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:20.539+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:20.805 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:20.802+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:21.554 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:21.552+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:21.772 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:21.770+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:22.585 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:22.583+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:22.757 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:22.755+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:23.632 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:23.630+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:23.804 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:23.802+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:24.607 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:24.605+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:24.787 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:24.785+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:25.605 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:25.603+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:25.815 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:25.813+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:26.581 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:26.579+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:26.794 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:26.792+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:27.601 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:27.599+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:27.829 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:27.827+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:28.571 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:28.569+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:28.782 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:28.781+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:29.538 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:29.536+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:29.751 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:29.749+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:30.588 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:30.586+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:30.764 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:30.762+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:31.578 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:31.576+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:31.789 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:31.787+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:32.533 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:32.531+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:32.743 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:32.741+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:33.528 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:33.526+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:33.717 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:33.715+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:34.483 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:34.481+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:34.745 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:34.743+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:35.515 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:35.513+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:35.750 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:35.748+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:36.494 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:36.493+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:36.711 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:36.709+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:37.465 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:37.463+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:37.704 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:37.702+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:38.505 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:38.502+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:38.700 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:38.697+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:39.529 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:39.526+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:39.720 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:39.717+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:40.481 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:40.478+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:40.747 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:40.744+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:41.443 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:41.440+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:41.712 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:41.709+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:42.437 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:42.434+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:42.697 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:42.694+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:43.483 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:43.480+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:43.704 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:43.701+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:44.454 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:44.451+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:44.731 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:44.728+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:45.439 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:45.436+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:45.732 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:45.729+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:46.484 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:46.481+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:46.735 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:46.733+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:47.489 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:47.486+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:47.746 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:47.743+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:48.442 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:48.439+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:48.762 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:48.760+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:49.403 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:49.400+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:49.738 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:49.735+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:50.374 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:50.372+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:50.728 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:50.726+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:51.355 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:51.352+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:51.698 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:51.695+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:52.313 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:52.310+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:52.678 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:52.675+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:53.279 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:53.276+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:53.634 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:53.631+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:54.231 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:54.228+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:54.649 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:54.646+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:55.241 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:55.238+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:55.651 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:55.648+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:56.253 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:56.250+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:56.690 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:56.688+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:57.241 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:57.238+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:57.654 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:57.651+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:58.283 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:58.280+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:58.617 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:58.614+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:59.306 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T16:59:59.303+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T16:59:59.617 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T16:59:59.614+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:00.309 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:00.306+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:00.583 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:00.580+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:01.313 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:01.310+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:01.563 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:01.560+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:02.351 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:02.348+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:02.556 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:02.553+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:03.399 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:03.396+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:03.566 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:03.564+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:04.422 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:04.419+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:04.605 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:04.602+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:05.396 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:05.393+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:05.604 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:05.601+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:06.371 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:06.368+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:06.557 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:06.554+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:07.382 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:07.379+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:07.523 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:07.520+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:08.355 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:08.352+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:08.542 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:08.539+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:09.364 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:09.361+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:09.564 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:09.561+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:10.395 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:10.392+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:10.540 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:10.538+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:11.404 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:11.401+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:11.509 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:11.506+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:12.388 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:12.386+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:12.522 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:12.520+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:13.355 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:13.353+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:13.513 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:13.511+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:14.311 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:14.309+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:14.561 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:14.559+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:15.304 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:15.302+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:15.572 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:15.570+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:16.261 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:16.259+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:16.581 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:16.580+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:17.290 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:17.286+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:17.609 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:17.607+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:18.291 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:18.288+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:18.654 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:18.652+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:19.267 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:19.266+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:19.624 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:19.622+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:20.272 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:20.270+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:20.614 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:20.612+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:21.287 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:21.286+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:21.609 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:21.607+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:22.287 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:22.285+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:22.572 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:22.570+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:23.321 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:23.319+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:23.572 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:23.570+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:24.306 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:24.304+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:24.619 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:24.617+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:25.276 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:25.274+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:25.629 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:25.628+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:26.238 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:26.236+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:26.673 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:26.671+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:27.262 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:27.260+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:27.651 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:27.650+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:28.277 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:28.275+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:28.618 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:28.616+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:29.294 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:29.292+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:29.651 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:29.648+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:30.247 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:30.244+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:30.694 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:30.691+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:31.296 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:31.293+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:31.690 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:31.687+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:32.303 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:32.300+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:32.712 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:32.709+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:33.320 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:33.317+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:33.689 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:33.685+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:34.323 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:34.320+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:34.668 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:34.665+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:35.360 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:35.357+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:35.682 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:35.679+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:36.399 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:36.396+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:36.644 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:36.641+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:37.443 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:37.440+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:37.668 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:37.665+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:38.446 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:38.443+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:38.683 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:38.680+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:39.488 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:39.485+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:39.724 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:39.721+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:40.442 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:40.440+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:40.724 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:40.721+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:41.415 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:41.412+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:41.724 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:41.721+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:42.400 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:42.397+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:42.683 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:42.680+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:43.411 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:43.408+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:43.671 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:43.668+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:44.381 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:44.378+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:44.642 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:44.639+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:45.424 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:45.421+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:45.650 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:45.647+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:46.376 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:46.373+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:46.637 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:46.634+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:47.361 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:47.358+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:47.642 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:47.639+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:48.329 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:48.326+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:48.617 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:48.614+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:49.334 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:49.331+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:49.610 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:49.607+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:50.353 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:50.350+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:50.577 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:50.574+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:51.346 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:51.343+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:51.531 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:51.528+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:52.385 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:52.382+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:52.550 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:52.547+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:53.394 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:53.391+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:53.574 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:53.571+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:54.372 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:54.369+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:54.559 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:54.556+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:55.365 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:55.362+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:55.605 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:55.602+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:56.322 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:56.320+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:56.591 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:56.588+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:57.317 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:57.314+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:57.615 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:57.613+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:58.306 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:58.303+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:58.619 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:58.616+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:59.264 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:00:59.261+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:00:59.631 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:00:59.629+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:00.290 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:00.287+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:00.664 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:00.661+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:01.269 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:01.267+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:01.711 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:01.709+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:02.288 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:02.285+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:02.695 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:02.692+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:03.283 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:03.280+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:03.707 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:03.705+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:04.262 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:04.260+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:04.712 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:04.710+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:05.219 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:05.217+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:05.677 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:05.675+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:06.232 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:06.230+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:06.667 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:06.665+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:07.252 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:07.250+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:07.618 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:07.617+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:08.227 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:08.225+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:08.574 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:08.572+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:09.231 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:09.229+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:09.594 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:09.592+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:10.262 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:10.260+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:10.576 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:10.574+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:11.262 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:11.260+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:11.549 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:11.547+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:12.304 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:12.303+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:12.554 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:12.552+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:13.294 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:13.292+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:13.593 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:13.591+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:14.305 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:14.303+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:14.613 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:14.611+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:15.339 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:15.337+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:15.599 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:15.597+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:16.353 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:16.351+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:16.602 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:16.601+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:17.336 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:17.335+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:17.583 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:17.581+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:18.329 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:18.327+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:18.561 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:18.559+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:19.294 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:19.292+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:19.609 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:19.607+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:20.291 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:20.289+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:20.571 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:20.569+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:21.263 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:21.261+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:21.552 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:21.549+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:22.284 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:22.281+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:22.537 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:22.534+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:23.278 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:23.275+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:23.527 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:23.524+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:24.329 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:24.326+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:24.531 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:24.528+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:25.355 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:25.352+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:25.523 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:25.512+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:26.389 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:26.386+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:26.535 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:26.533+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:27.357 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:27.354+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:27.559 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:27.556+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:28.342 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:28.340+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:28.543 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:28.540+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:29.300 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:29.297+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:29.553 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:29.550+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:30.260 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:30.257+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:30.540 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:30.538+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:31.252 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:31.249+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:31.511 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:31.508+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:32.243 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:32.240+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:32.546 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:32.543+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:33.230 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:33.228+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:33.590 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:33.587+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:34.240 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:34.237+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:34.554 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:34.551+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:35.224 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:35.221+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:35.597 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:35.592+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:36.237 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:36.234+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:36.563 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:36.560+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:37.231 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:37.228+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:37.574 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:37.571+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:38.241 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:38.238+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:38.548 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:38.545+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:39.244 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:39.241+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:39.590 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:39.588+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:40.277 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:40.274+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:40.560 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:40.557+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:41.250 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:41.247+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:41.608 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:41.606+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:42.224 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:42.221+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:42.655 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:42.652+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:43.204 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:43.202+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:43.679 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:43.676+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:44.170 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:44.167+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:44.687 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:44.684+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:45.219 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:45.216+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:45.644 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:45.642+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:46.195 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:46.192+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:46.635 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:46.632+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:47.197 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:47.194+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:47.647 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:47.644+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:48.168 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:48.165+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:48.686 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:48.683+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:49.200 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:49.198+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:49.648 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:49.646+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:50.170 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:50.168+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:50.605 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:50.602+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:51.168 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:51.165+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:51.594 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:51.592+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:52.189 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:52.186+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:52.619 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:52.617+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:53.224 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:53.221+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:53.573 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:53.570+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:54.238 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:54.235+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:54.538 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:54.535+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:55.269 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:55.267+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:55.495 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:55.493+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:56.258 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:56.256+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:56.448 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:56.446+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:57.220 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:57.218+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:57.497 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:57.495+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:58.214 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:58.212+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:58.504 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:58.502+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:59.250 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:01:59.248+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:01:59.492 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:01:59.490+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:00.281 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:00.279+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:00.525 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:00.523+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:01.318 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:01.316+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:01.556 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:01.554+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:02.290 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:02.288+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:02.569 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:02.567+0000 7fa30c07e700 -1 osd.0 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:03.248 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:03.246+0000 7f1661185700 -1 osd.1 151 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:03.610 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:03.608+0000 7fa30c07e700 -1 osd.0 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:04.218 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:04.217+0000 7f1661185700 -1 osd.1 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:04.585 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:04.583+0000 7fa30c07e700 -1 osd.0 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:05.265 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:05.263+0000 7f1661185700 -1 osd.1 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:05.577 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:05.575+0000 7fa30c07e700 -1 osd.0 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:06.227 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:06.225+0000 7f1661185700 -1 osd.1 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:06.624 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:06.622+0000 7fa30c07e700 -1 osd.0 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:07.246 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:07.244+0000 7f1661185700 -1 osd.1 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:07.597 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:07.595+0000 7fa30c07e700 -1 osd.0 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:08.264 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:08.262+0000 7f1661185700 -1 osd.1 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:08.614 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:08.612+0000 7fa30c07e700 -1 osd.0 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:09.295 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:09.293+0000 7f1661185700 -1 osd.1 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:09.594 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:09.592+0000 7fa30c07e700 -1 osd.0 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:10.302 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:10.300+0000 7f1661185700 -1 osd.1 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:10.558 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:10.556+0000 7fa30c07e700 -1 osd.0 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:11.259 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:11.258+0000 7f1661185700 -1 osd.1 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:11.543 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:11.541+0000 7fa30c07e700 -1 osd.0 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:12.260 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:12.258+0000 7f1661185700 -1 osd.1 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:12.585 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:12.583+0000 7fa30c07e700 -1 osd.0 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:13.238 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:13.235+0000 7f1661185700 -1 osd.1 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:13.586 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:13.583+0000 7fa30c07e700 -1 osd.0 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:14.232 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:14.229+0000 7f1661185700 -1 osd.1 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:14.579 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:14.577+0000 7fa30c07e700 -1 osd.0 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:15.253 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:15.250+0000 7f1661185700 -1 osd.1 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:15.538 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:15.535+0000 7fa30c07e700 -1 osd.0 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:16.269 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:16.267+0000 7f1661185700 -1 osd.1 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:16.506 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:16.503+0000 7fa30c07e700 -1 osd.0 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:17.243 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:17.240+0000 7f1661185700 -1 osd.1 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:17.547 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:17.544+0000 7fa30c07e700 -1 osd.0 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:18.196 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:18.193+0000 7f1661185700 -1 osd.1 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:18.547 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:18.545+0000 7fa30c07e700 -1 osd.0 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:19.178 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:19.175+0000 7f1661185700 -1 osd.1 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:19.537 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:19.535+0000 7fa30c07e700 -1 osd.0 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:20.137 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:20.134+0000 7f1661185700 -1 osd.1 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:20.492 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:20.489+0000 7fa30c07e700 -1 osd.0 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:21.106 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:21.103+0000 7f1661185700 -1 osd.1 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:21.505 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:21.502+0000 7fa30c07e700 -1 osd.0 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:22.140 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:22.137+0000 7f1661185700 -1 osd.1 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:22.500 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:22.497+0000 7fa30c07e700 -1 osd.0 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:23.176 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:23.173+0000 7f1661185700 -1 osd.1 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:23.486 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:23.483+0000 7fa30c07e700 -1 osd.0 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:24.216 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:24.213+0000 7f1661185700 -1 osd.1 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:24.480 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:24.478+0000 7fa30c07e700 -1 osd.0 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:25.211 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:25.208+0000 7f1661185700 -1 osd.1 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:25.520 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:25.517+0000 7fa30c07e700 -1 osd.0 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:26.167 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:26.164+0000 7f1661185700 -1 osd.1 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:26.533 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:26.531+0000 7fa30c07e700 -1 osd.0 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:27.153 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:27.150+0000 7f1661185700 -1 osd.1 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:27.539 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:27.536+0000 7fa30c07e700 -1 osd.0 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:28.149 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:28.146+0000 7f1661185700 -1 osd.1 152 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:28.535 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:28.532+0000 7fa30c07e700 -1 osd.0 153 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:29.166 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:29.163+0000 7f1661185700 -1 osd.1 153 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:29.545 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:29.542+0000 7fa30c07e700 -1 osd.0 153 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:30.140 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:30.137+0000 7f1661185700 -1 osd.1 153 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:30.538 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:30.535+0000 7fa30c07e700 -1 osd.0 153 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:31.149 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:31.146+0000 7f1661185700 -1 osd.1 153 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:31.505 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:31.502+0000 7fa30c07e700 -1 osd.0 153 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:32.105 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:32.099+0000 7f1661185700 -1 osd.1 153 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:32.523 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:32.520+0000 7fa30c07e700 -1 osd.0 153 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:33.098 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:33.095+0000 7f1661185700 -1 osd.1 153 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:33.539 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:33.536+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:34.053 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:34.051+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:34.559 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:34.556+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:35.010 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:35.007+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:35.540 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:35.538+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:36.032 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:36.029+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:36.562 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:36.559+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:36.987 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:36.984+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:37.609 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:37.606+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:37.945 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:37.942+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:38.616 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:38.613+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:38.969 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:38.966+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:39.575 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:39.573+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:39.962 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:39.959+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:40.626 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:40.623+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:40.970 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:40.967+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:41.609 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:41.606+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:41.988 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:41.985+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:42.573 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:42.570+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:43.015 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:43.012+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:43.588 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:43.585+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:43.971 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:43.968+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:44.599 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:44.597+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:44.973 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:44.970+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:45.559 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:45.556+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:45.938 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:45.935+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:46.606 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:46.604+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:46.909 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:46.907+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:47.620 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:47.618+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:47.946 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:47.945+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:48.583 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:48.581+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:48.988 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:48.986+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:49.559 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:49.557+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:50.033 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:50.031+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:50.589 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:50.587+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:51.004 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:51.002+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:51.597 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:51.595+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:52.006 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:52.005+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:52.596 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:52.594+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:53.052 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:53.050+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:53.593 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:53.591+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:54.083 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:54.081+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:54.592 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:54.590+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:55.102 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:55.100+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:55.593 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:55.591+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:56.149 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:56.147+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:56.623 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:56.621+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:57.149 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:57.147+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:57.714 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:57.660+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:58.123 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:58.121+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:58.681 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:58.679+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:59.091 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:02:59.089+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:02:59.649 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:02:59.647+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:00.051 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:00.049+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:00.630 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:00.628+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:01.016 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:01.015+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:01.652 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:01.650+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:02.011 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:02.009+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:02.653 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:02.651+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:03.062 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:03.060+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:03.641 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:03.639+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:04.038 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:04.037+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:04.610 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:04.608+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:05.071 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:05.068+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:05.645 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:05.642+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:06.108 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:06.106+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:06.623 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:06.620+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:07.130 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:07.127+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:07.584 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:07.582+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:08.118 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:08.115+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:08.608 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:08.606+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:09.125 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:09.122+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:09.129 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T17:03:09.127+0000 7fd64ac56700 0 -- 172.21.15.82:0/3601686610 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x56278280c820 msgr2=0x7fd630038ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-18T17:03:09.631 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:09.628+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:10.130 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:10.127+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:10.623 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:10.620+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:11.176 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:11.172+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:11.643 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:11.640+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:12.157 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:12.154+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:12.676 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:12.673+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:13.173 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:13.170+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:13.702 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:13.699+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:14.146 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:14.143+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:14.684 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:14.681+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:15.167 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:15.164+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:15.691 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:15.688+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:16.186 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:16.184+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:16.730 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:16.727+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:17.218 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:17.215+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:17.734 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:17.732+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:18.190 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:18.188+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:18.780 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:18.777+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:19.143 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:19.140+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:19.792 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:19.789+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:20.183 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:20.180+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:20.835 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:20.832+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:21.180 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:21.177+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:21.806 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:21.804+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:22.194 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:22.191+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:22.846 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:22.843+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:23.209 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:23.206+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:23.827 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:23.824+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:24.203 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:24.200+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:24.835 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:24.832+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:25.201 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:25.198+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:25.852 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:25.849+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:26.176 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:26.174+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:26.884 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:26.881+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:27.169 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:27.167+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:27.913 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:27.910+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:28.138 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:28.135+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:28.876 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:28.873+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:29.163 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:29.160+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:29.848 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:29.845+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:30.194 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:30.191+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:30.836 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:30.834+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:31.230 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:31.227+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:31.827 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:31.824+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:32.217 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:32.214+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:32.825 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:32.822+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:33.219 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:33.216+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:33.798 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:33.795+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:34.203 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:34.200+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:34.836 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:34.833+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:35.203 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:35.200+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:35.869 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:35.866+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:36.194 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:36.191+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:36.897 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:36.894+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:37.201 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:37.198+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:37.905 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:37.903+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:38.162 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:38.160+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:38.936 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:38.934+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:39.212 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:39.209+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:39.977 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:39.976+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:40.231 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:40.229+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:40.934 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:40.933+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:41.264 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:41.262+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:41.903 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:41.901+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:42.265 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:42.263+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:42.899 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:42.897+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:43.264 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:43.262+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:43.871 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:43.870+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:44.313 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:44.311+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:44.823 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:44.821+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:45.300 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:45.298+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:45.783 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:45.781+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:46.283 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:46.282+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:46.734 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:46.732+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:47.235 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:47.234+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:47.743 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:47.742+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:48.226 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:48.224+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:48.781 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:48.780+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:49.257 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:49.255+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:49.757 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:49.753+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:50.222 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:50.220+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:50.713 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:50.712+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:51.174 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:51.172+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:51.733 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:51.732+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:52.208 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:52.206+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:52.738 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:52.736+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:53.159 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:53.157+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:53.755 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:53.753+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:54.133 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:54.131+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:54.707 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:54.705+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:55.164 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:55.162+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:55.695 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:55.694+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:56.192 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:56.190+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:56.698 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:56.696+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:57.149 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:57.147+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:57.712 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:57.709+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:58.133 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:58.130+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:58.697 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:58.694+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:59.093 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:03:59.090+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:03:59.658 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:03:59.655+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:00.059 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:00.057+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:00.703 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:00.701+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:01.016 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:01.013+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:01.714 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:01.712+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:02.049 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:02.047+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:02.756 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:02.753+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:03.044 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:03.041+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:03.764 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:03.761+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:04.014 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:04.011+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:04.739 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:04.737+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:04.982 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:04.979+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:05.731 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:05.729+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:05.941 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:05.938+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:06.702 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:06.699+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:06.926 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:06.923+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:07.672 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:07.669+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:07.975 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:07.973+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:08.646 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:08.643+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:08.961 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:08.959+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:09.683 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:09.681+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:09.962 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:09.959+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:10.635 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:10.632+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:10.988 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:10.985+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:11.652 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:11.649+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:11.960 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:11.957+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:12.638 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:12.635+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:12.994 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:12.991+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:13.593 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:13.590+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:14.016 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:14.013+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:14.569 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:14.566+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:14.972 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:14.970+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:15.597 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:15.594+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:15.923 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:15.921+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:16.609 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:16.606+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:16.945 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:16.942+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:17.597 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:17.594+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:17.992 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:17.989+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:18.562 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:18.559+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:18.944 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:18.941+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:19.607 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:19.604+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:19.901 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:19.898+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:20.628 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:20.625+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:20.902 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:20.899+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:21.672 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:21.669+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:21.914 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:21.911+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:22.670 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:22.667+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:22.940 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:22.937+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:23.630 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:23.627+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:23.956 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:23.954+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:24.630 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:24.627+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:24.984 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:24.981+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:25.597 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:25.594+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:26.034 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:26.031+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:26.643 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:26.641+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:27.030 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:27.028+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:27.622 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:27.619+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:28.073 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:28.070+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:28.579 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:28.577+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:29.035 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:29.033+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:29.574 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:29.571+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:29.988 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:29.986+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:30.602 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:30.599+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:31.009 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:31.006+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:31.596 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:31.595+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:31.990 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:31.988+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:32.645 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:32.643+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:33.021 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:33.017+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:33.663 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:33.661+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:34.017 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:34.015+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:34.668 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:34.666+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:34.977 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:34.975+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:35.715 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:35.713+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:36.009 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:36.007+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:36.759 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:36.757+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:37.005 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:37.003+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:37.807 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:37.805+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:37.967 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:37.965+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:38.854 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:38.852+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:38.943 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:38.941+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:39.833 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:39.831+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:39.978 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:39.976+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:40.855 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:40.853+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:40.931 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:40.929+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:41.808 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:41.806+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:41.964 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:41.962+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:42.828 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:42.825+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:42.966 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:42.964+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:43.795 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:43.793+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:43.922 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:43.921+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:44.841 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:44.839+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:44.897 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:44.894+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:45.830 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:45.828+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:45.870 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:45.869+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:46.861 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:46.859+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:46.921 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:46.919+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:47.834 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:47.832+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:47.937 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:47.936+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:48.813 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:48.812+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:48.928 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:48.926+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:49.770 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:49.767+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:49.925 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:49.923+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:50.818 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:50.815+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:50.939 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:50.937+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:51.825 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:51.823+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:51.980 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:51.977+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:52.859 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:52.856+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:52.953 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:52.950+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:53.869 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:53.866+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:53.937 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:53.934+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:54.883 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:54.880+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:54.981 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:54.978+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:55.874 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:55.871+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:55.966 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:55.963+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:56.866 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:56.863+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:56.968 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:56.965+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:57.904 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:57.901+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:57.959 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:57.956+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:58.930 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:58.928+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:58.989 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:04:58.986+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:04:59.955 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:04:59.952+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:00.035 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:00.032+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:00.920 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:00.917+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:01.060 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:01.058+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:01.907 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:01.904+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:02.108 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:02.105+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:02.941 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:02.939+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:03.111 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:03.108+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:03.985 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:03.982+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:04.123 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:04.120+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:04.965 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:04.962+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:05.143 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:05.140+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:05.981 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:05.978+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:06.121 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:06.118+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:07.018 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:07.015+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:07.148 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:07.144+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:08.038 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:08.035+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:08.098 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:08.095+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:09.051 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:09.048+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:09.062 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:09.060+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:10.058 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:10.055+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:10.063 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:10.061+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:11.022 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:11.019+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:11.069 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:11.067+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:11.984 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:11.981+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:12.103 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:12.101+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:12.958 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:12.955+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:13.133 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:13.130+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:13.920 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:13.918+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:14.121 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:14.119+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:14.891 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:14.888+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:15.127 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:15.124+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:15.901 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:15.897+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:16.118 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:16.116+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:16.884 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:16.881+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:17.091 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:17.088+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:17.922 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:17.919+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:18.082 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:18.079+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:18.879 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:18.876+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:19.040 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:19.037+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:19.842 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:19.839+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:19.994 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:19.991+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:20.801 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:20.799+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:20.967 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:20.964+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:21.800 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:21.797+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:21.977 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:21.974+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:22.755 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:22.752+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:22.994 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:22.991+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:23.795 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:23.793+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:23.953 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:23.951+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:24.768 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:24.766+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:24.932 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:24.930+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:25.793 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:25.791+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:25.901 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:25.898+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:26.821 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:26.819+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:26.923 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:26.921+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:27.868 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:27.866+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:27.888 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:27.886+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:28.889 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:28.887+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:28.903 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:28.902+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:29.927 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:29.926+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:29.942 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:29.941+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:30.910 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:30.908+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:30.916 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:30.914+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:31.895 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:31.893+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:31.907 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:31.905+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:32.924 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:32.922+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:32.938 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:32.936+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:33.894 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:33.892+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:33.922 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:33.920+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:34.880 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:34.878+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:34.907 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:34.905+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:35.871 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:35.869+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:35.902 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:35.900+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:36.886 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:36.884+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:36.952 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:36.950+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:37.913 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:37.911+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:37.995 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:37.993+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:38.923 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:38.922+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:39.044 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:39.042+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:39.958 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:39.957+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:40.018 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:40.017+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:40.931 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:40.929+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:41.042 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:41.040+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:41.949 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:41.946+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:42.087 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:42.085+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:42.930 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:42.927+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:43.059 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:43.056+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:43.916 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:43.913+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:44.077 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:44.074+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:44.885 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:44.882+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:45.125 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:45.122+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:45.934 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:45.931+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:46.157 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:46.155+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:46.947 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:46.944+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:47.147 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:47.144+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:47.975 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:47.972+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:48.193 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:48.191+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:49.020 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:49.017+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:49.166 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:49.163+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:50.045 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:50.042+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:50.205 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:50.202+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:51.015 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:51.012+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:51.209 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:51.206+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:52.064 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:52.062+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:52.231 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:52.229+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:53.044 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:53.041+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:53.273 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:53.271+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:54.080 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:54.077+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:54.273 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:54.270+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:55.045 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:55.043+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:55.284 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:55.281+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:56.070 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:56.067+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:56.268 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:56.265+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:57.120 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:57.117+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:57.289 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:57.286+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:58.111 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:58.109+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:58.327 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:58.324+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:59.133 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:05:59.131+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:05:59.333 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:05:59.330+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:00.171 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:00.168+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:00.344 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:00.342+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:01.162 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:01.159+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:01.366 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:01.363+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:02.136 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:02.133+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:02.402 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:02.399+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:03.185 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:03.183+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:03.374 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:03.371+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:04.235 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:04.233+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:04.424 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:04.421+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:05.191 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:05.188+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:05.383 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:05.381+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:06.189 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:06.186+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:06.343 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:06.340+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:07.203 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:07.200+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:07.341 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:07.339+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:08.162 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:08.159+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:08.295 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:08.292+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:09.169 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:09.166+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:09.301 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:09.298+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:10.181 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:10.179+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:10.329 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:10.326+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:11.146 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:11.143+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:11.311 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:11.309+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:12.104 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:12.101+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:12.304 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:12.301+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:13.113 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:13.110+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:13.301 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:13.297+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:14.152 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:14.149+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:14.308 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:14.306+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:15.158 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:15.155+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:15.357 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:15.354+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:16.175 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:16.173+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:16.325 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:16.323+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:17.159 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:17.157+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:17.323 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:17.321+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:18.205 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:18.203+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:18.319 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:18.318+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:19.175 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:19.173+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:19.303 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:19.301+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:20.209 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:20.208+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:20.297 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:20.295+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:21.188 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:21.187+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:21.340 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:21.338+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:22.180 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:22.179+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:22.374 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:22.372+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:23.224 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:23.223+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:23.369 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:23.367+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:24.197 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:24.195+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:24.406 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:24.405+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:25.227 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:25.225+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:25.409 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:25.407+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:26.217 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:26.215+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:26.425 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:26.423+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:27.257 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:27.255+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:27.464 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:27.463+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:28.232 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:28.231+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:28.493 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:28.491+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:29.223 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:29.221+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:29.474 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:29.472+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:30.233 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:30.231+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:30.444 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:30.443+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:31.202 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:31.201+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:31.427 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:31.425+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:32.209 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:32.207+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:32.419 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:32.417+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:33.168 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:33.166+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:33.416 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:33.414+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:34.191 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:34.188+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:34.389 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:34.386+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:35.165 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:35.162+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:35.401 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:35.398+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:36.134 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:36.131+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:36.420 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:36.417+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:37.165 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:37.162+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:37.467 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:37.464+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:38.150 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:38.148+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:38.449 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:38.446+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:39.111 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:39.109+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:39.468 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:39.465+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:40.139 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:40.137+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:40.432 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:40.429+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:41.129 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:41.126+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:41.420 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:41.418+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:42.164 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:42.161+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:42.441 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:42.438+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:43.152 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:43.149+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:43.453 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:43.450+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:44.109 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:44.106+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:44.425 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:44.423+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:45.125 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:45.122+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:45.415 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:45.412+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:46.159 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:46.156+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:46.403 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:46.401+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:47.191 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:47.188+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:47.399 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:47.397+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:48.144 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:48.141+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:48.355 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:48.353+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:49.150 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:49.147+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:49.356 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:49.353+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:50.103 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:50.101+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:50.384 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:50.381+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:51.131 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:51.128+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:51.380 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:51.377+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:52.104 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:52.101+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:52.399 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:52.396+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:53.060 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:53.057+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:53.404 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:53.401+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:54.097 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:54.094+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:54.366 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:54.363+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:55.115 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:55.112+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:55.383 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:55.380+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:56.076 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:56.073+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:56.426 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:56.423+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:57.062 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:57.059+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:57.417 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:57.414+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:58.041 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:58.038+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:58.390 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:58.387+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:59.082 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:06:59.079+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:06:59.434 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:06:59.431+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:00.063 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:00.061+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:00.478 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:00.475+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:01.032 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:01.030+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:01.484 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:01.481+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:01.994 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:01.991+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:02.484 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:02.481+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:02.974 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:02.972+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:03.507 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:03.505+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:04.013 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:04.010+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:04.531 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:04.528+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:04.999 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:04.996+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:05.530 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:05.527+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:06.004 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:06.001+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:06.560 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:06.557+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:07.040 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:07.038+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:07.537 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:07.535+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:08.080 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:08.077+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:08.531 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:08.529+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:09.081 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:09.079+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:09.498 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:09.496+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:10.073 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:10.071+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:10.482 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:10.481+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:11.076 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:11.074+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:11.442 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:11.440+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:12.041 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:12.039+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:12.407 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:12.405+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:13.012 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:13.010+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:13.438 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:13.436+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:14.039 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:14.037+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:14.397 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:14.395+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:15.060 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:15.059+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:15.419 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:15.418+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:16.013 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:16.012+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:16.467 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:16.465+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:17.002 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:17.000+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:17.484 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:17.482+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:17.998 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:17.996+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:18.467 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:18.465+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:18.967 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:18.966+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:19.475 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:19.473+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:19.959 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:19.957+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:20.437 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:20.435+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:20.970 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:20.969+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:21.394 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:21.392+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:22.003 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:22.001+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:22.422 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:22.420+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:23.024 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:23.023+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:23.462 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:23.460+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:24.045 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:24.043+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:24.430 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:24.428+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:25.001 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:24.999+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:25.442 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:25.441+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:26.031 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:26.030+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:26.407 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:26.404+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:27.038 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:27.035+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:27.446 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:27.444+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:28.046 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:28.044+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:28.416 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:28.413+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:29.025 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:29.023+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:29.441 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:29.439+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:29.982 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:29.980+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:30.490 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:30.487+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:30.968 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:30.966+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:31.481 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:31.478+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:31.954 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:31.952+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:32.526 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:32.523+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:32.990 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:32.988+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:33.512 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:33.509+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:33.941 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:33.939+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:34.549 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:34.546+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:34.934 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:34.932+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:35.547 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:35.545+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:35.963 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:35.960+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:36.543 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:36.541+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:36.925 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:36.922+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:37.563 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:37.560+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:37.942 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:37.940+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:38.549 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:38.547+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:38.894 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:38.892+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:39.573 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:39.570+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:39.854 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:39.852+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:40.526 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:40.523+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:40.813 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:40.810+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:41.572 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:41.569+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:41.780 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:41.778+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:42.556 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:42.553+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:42.748 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:42.745+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:43.584 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:43.580+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:43.737 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:43.734+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:44.626 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:44.623+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:44.746 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:44.744+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:45.634 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:45.631+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:45.730 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:45.728+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:46.598 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:46.595+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:46.758 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:46.756+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:47.613 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:47.610+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:47.714 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:47.711+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:48.631 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:48.628+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:48.742 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:48.740+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:49.659 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:49.656+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:49.729 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:49.726+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:50.676 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:50.673+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:50.764 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:50.762+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:51.638 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:51.635+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:51.762 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:51.759+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:52.634 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:52.632+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:52.731 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:52.728+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:53.672 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:53.670+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:53.724 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:53.722+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:54.666 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:54.664+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:54.756 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:54.753+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:55.623 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:55.620+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:55.724 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:55.721+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:56.653 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:56.650+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:56.708 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:56.706+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:57.622 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:57.619+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:57.741 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:57.738+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:58.605 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:58.603+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:58.787 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:58.784+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:59.652 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:07:59.650+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:07:59.787 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:07:59.784+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:00.641 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:00.639+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:00.759 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:00.758+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:01.686 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:01.684+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:01.712 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:01.710+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:02.653 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:02.651+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:02.699 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:02.697+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:03.653 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:03.651+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:03.746 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:03.744+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:04.700 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:04.698+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:04.725 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:04.724+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:05.655 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:05.654+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:05.702 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:05.700+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:06.651 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:06.650+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:06.727 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:06.726+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:07.627 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:07.625+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:07.767 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:07.765+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:08.644 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:08.642+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:08.723 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:08.722+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:09.635 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:09.634+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:09.675 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:09.674+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:10.645 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:10.643+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:10.693 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:10.692+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:11.632 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:11.630+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:11.726 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:11.724+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:12.651 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:12.649+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:12.768 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:12.766+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:13.653 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:13.652+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:13.796 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:13.794+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:14.635 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:14.633+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:14.791 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:14.790+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:15.604 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:15.602+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:15.777 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:15.775+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:16.647 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:16.645+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:16.742 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:16.740+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:17.622 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:17.621+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:17.711 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:17.709+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:18.668 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:18.667+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:18.754 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:18.752+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:19.697 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:19.695+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:19.707 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:19.705+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:20.652 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:20.649+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:20.728 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:20.725+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:21.646 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:21.643+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:21.695 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:21.693+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:22.678 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:22.675+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:22.690 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:22.688+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:23.724 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:23.721+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:23.739 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:23.737+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:24.743 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:24.740+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:24.789 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:24.786+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:25.787 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:25.784+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:25.819 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:25.816+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:26.794 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:26.791+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:26.867 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:26.865+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:27.751 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:27.748+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:27.861 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:27.859+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:28.744 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:28.741+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:28.881 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:28.879+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:29.751 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:29.748+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:29.865 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:29.862+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:30.799 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:30.796+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:30.898 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:30.895+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:31.826 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:31.823+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:31.904 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:31.901+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:32.872 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:32.870+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:32.898 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:32.895+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:33.898 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:33.895+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:33.911 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:33.908+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:34.918 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:34.915+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:34.933 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:34.931+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:35.883 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:35.880+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:35.945 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:35.942+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:36.897 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:36.895+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:36.915 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:36.912+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:37.870 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:37.868+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:37.942 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:37.939+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:38.829 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:38.827+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:38.934 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:38.932+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:39.813 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:39.811+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:39.926 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:39.923+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:40.765 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:40.762+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:40.907 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:40.905+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:41.785 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:41.782+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:41.911 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:41.908+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:42.827 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:42.824+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:42.900 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:42.897+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:43.798 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:43.795+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:43.852 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:43.849+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:44.753 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:44.751+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:44.855 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:44.853+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:45.741 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:45.738+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:45.845 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:45.842+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:46.739 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:46.736+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:46.895 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:46.893+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:47.728 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:47.725+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:47.866 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:47.863+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:48.705 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:48.702+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:48.865 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:48.862+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:49.715 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:49.712+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:49.899 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:49.896+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:50.719 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:50.716+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:50.925 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:50.923+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:51.711 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:51.709+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:51.915 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:51.912+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:52.669 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:52.667+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:52.894 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:52.891+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:53.637 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:53.635+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:53.902 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:53.900+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:54.644 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:54.643+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:54.867 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:54.865+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:55.635 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:55.633+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:55.874 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:55.872+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:56.616 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:56.614+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:56.912 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:56.911+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:57.618 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:57.617+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:57.952 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:57.951+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:58.581 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:58.579+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:58.972 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:58.971+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:59.590 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:08:59.589+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:08:59.955 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:08:59.953+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:00.585 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:00.583+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:00.968 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:00.967+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:01.550 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:01.549+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:01.977 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:01.975+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:02.550 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:02.549+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:02.951 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:02.949+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:03.549 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:03.547+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:03.936 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:03.934+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:04.518 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:04.515+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:04.960 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:04.959+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:05.565 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:05.563+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:05.919 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:05.917+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:06.613 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:06.612+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:06.906 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:06.905+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:07.577 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:07.575+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:07.888 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:07.887+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:08.627 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:08.625+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:08.908 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:08.906+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:09.605 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:09.603+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:09.891 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:09.889+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:10.583 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:10.581+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:10.910 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:10.909+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:11.587 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:11.585+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:11.891 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:11.888+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:12.615 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:12.613+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:12.896 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:12.894+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:13.653 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:13.650+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:13.922 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:13.919+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:14.688 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:14.685+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:14.919 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:14.917+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:15.695 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:15.692+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:15.942 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:15.940+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:16.729 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:16.726+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:16.918 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:16.916+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:17.781 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:17.776+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:17.916 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:17.914+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:18.815 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:18.812+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:18.899 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:18.897+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:19.840 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:19.837+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:19.891 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:19.889+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:20.824 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:20.822+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:20.931 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:20.928+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:21.781 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:21.778+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:21.908 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:21.905+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:22.739 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:22.737+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:22.859 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:22.856+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:23.774 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:23.772+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:23.848 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:23.846+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:24.781 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:24.778+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:24.820 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:24.817+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:25.767 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:25.764+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:25.782 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:25.780+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:26.761 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:26.759+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:26.832 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:26.829+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:27.771 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:27.769+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:27.859 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:27.856+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:28.808 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:28.805+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:28.858 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:28.855+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:29.768 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:29.766+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:29.902 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:29.899+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:30.780 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:30.778+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:30.908 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:30.906+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:31.732 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:31.729+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:31.892 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:31.889+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:32.760 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:32.757+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:32.877 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:32.874+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:33.727 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:33.724+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:33.845 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:33.843+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:34.726 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:34.724+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:34.808 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:34.806+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:35.697 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:35.694+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:35.804 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:35.802+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:36.671 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:36.668+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:36.834 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:36.832+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:37.687 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:37.684+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:37.816 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:37.813+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:38.724 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:38.722+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:38.785 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:38.783+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:39.710 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:39.707+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:39.813 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:39.811+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:40.664 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:40.661+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:40.804 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:40.801+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:41.664 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:41.662+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:41.813 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:41.810+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:42.685 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:42.669+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:42.819 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:42.816+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:43.636 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:43.634+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:43.772 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:43.770+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:44.626 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:44.623+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:44.759 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:44.757+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:45.612 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:45.611+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:45.766 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:45.764+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:46.654 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:46.652+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:46.794 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:46.792+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:47.639 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:47.637+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:47.800 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:47.798+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:48.650 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:48.648+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:48.815 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:48.814+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:49.643 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:49.641+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:49.785 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:49.783+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:50.637 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:50.634+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:50.754 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:50.753+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:51.653 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:51.651+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:51.764 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:51.762+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:52.605 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:52.603+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:52.749 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:52.748+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:53.624 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:53.623+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:53.761 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:53.760+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:54.591 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:54.589+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:54.806 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:54.805+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:55.630 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:55.628+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:55.839 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:55.838+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:56.601 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:56.600+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:56.855 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:56.853+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:57.575 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:57.574+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:57.843 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:57.842+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:58.530 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:58.528+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:58.850 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:58.848+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:59.489 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:09:59.488+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:09:59.823 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:09:59.821+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:00.473 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:00.471+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:00.870 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:00.868+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:01.450 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:01.449+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:01.880 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:01.878+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:02.485 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:02.484+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:02.879 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:02.877+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:03.500 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:03.499+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:03.912 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:03.909+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:04.548 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:04.545+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:04.948 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:04.945+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:05.575 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:05.573+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:05.974 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:05.971+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:06.620 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:06.617+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:07.021 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:07.019+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:07.668 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:07.665+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:08.052 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:08.049+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:08.718 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:08.715+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:09.079 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:09.077+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:09.710 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:09.708+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:10.061 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:10.059+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:10.693 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:10.691+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:11.084 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:11.081+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:11.669 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:11.666+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:12.075 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:12.072+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:12.708 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:12.706+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:13.064 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:13.061+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:13.720 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:13.717+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:14.027 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:14.025+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:14.748 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:14.746+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:14.998 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:14.995+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:15.705 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:15.703+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:16.013 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:16.011+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:16.729 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:16.727+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:17.031 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:17.028+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:17.715 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:17.712+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:18.034 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:18.032+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:18.673 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:18.670+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:19.024 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:19.021+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:19.712 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:19.710+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:19.978 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:19.976+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:20.752 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:20.749+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:20.934 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:20.932+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:21.732 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:21.729+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:21.912 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:21.910+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:22.689 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:22.687+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:22.886 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:22.884+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:23.656 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:23.653+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:23.915 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:23.912+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:24.610 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:24.607+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:24.927 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:24.924+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:25.619 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:25.617+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:25.887 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:25.885+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:26.588 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:26.586+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:26.914 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:26.912+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:27.633 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:27.631+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:27.949 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:27.946+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:28.640 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:28.637+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:28.968 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:28.966+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:29.655 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:29.652+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:29.937 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:29.935+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:30.617 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:30.614+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:30.911 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:30.909+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:31.609 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:31.606+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:31.960 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:31.957+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:32.628 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:32.625+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:32.947 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:32.944+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:33.654 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:33.652+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:33.967 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:33.964+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:34.633 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:34.630+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:34.977 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:34.975+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:35.609 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:35.606+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:35.991 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:35.988+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:36.561 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:36.559+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:37.005 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:37.002+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:37.530 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:37.528+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:38.018 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:38.017+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:38.513 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:38.512+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:39.009 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:39.007+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:39.498 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:39.496+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:39.973 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:39.971+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:40.546 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:40.545+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:40.925 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:40.923+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:41.530 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:41.529+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:41.940 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:41.938+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:42.507 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:42.505+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:42.909 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:42.907+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:43.537 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:43.535+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:43.925 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:43.923+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:44.493 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:44.492+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:44.922 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:44.921+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:45.459 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:45.457+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:45.954 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:45.952+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:46.464 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:46.462+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:46.985 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:46.983+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:47.497 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:47.495+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:47.975 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:47.973+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:48.528 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:48.527+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:49.023 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:49.021+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:49.550 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:49.548+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:50.011 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:50.010+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:50.559 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:50.557+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:50.998 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:50.997+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:51.580 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:51.579+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:51.986 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:51.984+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:52.534 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:52.532+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:53.013 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:53.012+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:53.496 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:53.495+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:54.054 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:54.052+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:54.454 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:54.453+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:55.039 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:55.037+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:55.500 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:55.499+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:56.008 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:56.007+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:56.519 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:56.516+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:57.041 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:57.039+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:57.490 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:57.487+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:58.065 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:58.062+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:58.507 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:58.504+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:59.034 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:10:59.032+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:10:59.540 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:10:59.537+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:00.016 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:00.013+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:00.588 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:00.586+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:01.050 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:01.047+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:01.604 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:01.602+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:02.006 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:02.004+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:02.620 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:02.618+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:03.014 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:03.011+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:03.664 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:03.662+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:04.043 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:04.040+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:04.686 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:04.684+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:05.006 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:05.003+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:05.726 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:05.724+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:05.979 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:05.977+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:06.736 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:06.734+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:06.980 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:06.977+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:07.770 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:07.767+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:08.011 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:08.008+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:08.747 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:08.745+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:08.998 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:08.996+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:09.791 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:09.788+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:09.996 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:09.994+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:10.758 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:10.755+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:11.009 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:11.006+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:11.804 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:11.801+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:12.021 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:12.019+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:12.767 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:12.765+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:13.062 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:13.060+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:13.767 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:13.765+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:14.043 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:14.040+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:14.749 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:14.746+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:15.065 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:15.062+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:15.744 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:15.741+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:16.096 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:16.093+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:16.732 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:16.729+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:17.075 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:17.072+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:17.780 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:17.778+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:18.031 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:18.028+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:18.775 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:18.773+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:19.038 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:19.035+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:19.814 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:19.812+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:20.042 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:20.040+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:20.766 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:20.764+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:21.055 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:21.052+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:21.793 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:21.791+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:22.086 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:22.084+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:22.789 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:22.787+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:23.047 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:23.045+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:23.787 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:23.785+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:24.090 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:24.088+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:24.775 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:24.772+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:25.058 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:25.056+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:25.736 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:25.734+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:26.018 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:26.016+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:26.764 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:26.761+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:26.987 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:26.984+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:27.804 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:27.802+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:28.011 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:28.009+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:28.853 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:28.850+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:29.011 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:29.008+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:29.846 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:29.844+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:29.980 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:29.977+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:30.829 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:30.827+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:30.931 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:30.930+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:31.809 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:31.805+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:31.970 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:31.968+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:32.792 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:32.790+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:32.921 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:32.919+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:33.805 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:33.804+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:33.891 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:33.890+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:34.777 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:34.776+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:34.921 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:34.920+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:35.754 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:35.753+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:35.917 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:35.916+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:36.760 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:36.759+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:36.915 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:36.913+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:37.725 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:37.724+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:37.921 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:37.919+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:38.727 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:38.726+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:38.886 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:38.885+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:39.741 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:39.740+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:39.860 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:39.859+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:40.786 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:40.785+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:40.818 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:40.817+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:41.769 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:41.767+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:41.854 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:41.852+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:42.775 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:42.773+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:42.857 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:42.855+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:43.806 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:43.804+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:43.831 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:43.829+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:44.775 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:44.774+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:44.800 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:44.798+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:45.771 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:45.769+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:45.773 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:45.772+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:46.774 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:46.773+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:46.779 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:46.778+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:47.728 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:47.727+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:47.733 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:47.732+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:48.685 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:48.683+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:48.721 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:48.719+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:49.675 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:49.672+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:49.698 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:49.695+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:50.665 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:50.663+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:50.746 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:50.743+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:51.688 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:51.685+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:51.712 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:51.709+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:52.684 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:52.682+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:52.743 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:52.741+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:53.692 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:53.690+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:53.780 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:53.777+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:54.670 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:54.668+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:54.731 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:54.728+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:55.678 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:55.675+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:55.700 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:55.697+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:56.678 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:56.676+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:56.703 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:56.700+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:57.655 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:57.653+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:57.703 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:57.700+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:58.666 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:58.663+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:58.680 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:58.678+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:59.642 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:11:59.639+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:11:59.692 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:11:59.689+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:00.648 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:00.645+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:00.708 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:00.706+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:01.698 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:01.696+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:01.699 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:01.697+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:02.683 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:02.680+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:02.691 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:02.689+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:03.654 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:03.652+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:03.691 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:03.688+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:04.664 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:04.661+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:04.671 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:04.669+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:05.623 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:05.620+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:05.625 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:05.622+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:06.587 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:06.584+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:06.596 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:06.594+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:07.563 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:07.561+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:07.601 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:07.599+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:08.589 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:08.587+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:08.603 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:08.601+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:09.622 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:09.619+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:09.650 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:09.647+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:10.633 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:10.631+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:10.698 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:10.696+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:11.602 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:11.600+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:11.744 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:11.742+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:12.651 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:12.648+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:12.780 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:12.777+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:13.608 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:13.606+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:13.772 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:13.769+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:14.633 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:14.631+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:14.756 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:14.753+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:15.659 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:15.657+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:15.743 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:15.740+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:16.695 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:16.692+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:16.726 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:16.724+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:17.701 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:17.698+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:17.730 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:17.728+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:18.695 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:18.692+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:18.711 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:18.708+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:19.689 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:19.686+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:19.713 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:19.710+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:20.664 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:20.662+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:20.687 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:20.685+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:21.627 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:21.624+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:21.676 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:21.673+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:22.615 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:22.613+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:22.638 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:22.637+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:23.582 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:23.580+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:23.644 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:23.643+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:24.540 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:24.539+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:24.645 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:24.643+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:25.560 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:25.559+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:25.646 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:25.645+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:26.611 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:26.609+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:26.625 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:26.623+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:27.569 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:27.567+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:27.657 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:27.655+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:28.551 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:28.549+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:28.642 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:28.641+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:29.508 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:29.506+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:29.672 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:29.671+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:30.556 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:30.554+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:30.656 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:30.655+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:31.548 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:31.547+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:31.662 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:31.661+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:32.513 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:32.511+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:32.712 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:32.711+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:33.560 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:33.558+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:33.743 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:33.742+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:34.525 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:34.524+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:34.739 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:34.738+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:35.551 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:35.550+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:35.701 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:35.700+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:36.584 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:36.583+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:36.718 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:36.717+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:37.578 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:37.577+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:37.756 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:37.755+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:38.615 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:38.613+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:38.710 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:38.708+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:39.574 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:39.572+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:39.710 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:39.707+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:40.594 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:40.592+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:40.752 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:40.750+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:41.624 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:41.621+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:41.753 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:41.751+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:42.616 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:42.614+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:42.728 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:42.726+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:43.641 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:43.638+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:43.731 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:43.728+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:44.659 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:44.657+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:44.705 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:44.702+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:45.709 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:45.707+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:45.716 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:45.714+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:46.724 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:46.722+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:46.736 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:46.734+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:47.710 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:47.707+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:47.765 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:47.762+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:48.726 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:48.723+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:48.736 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:48.734+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:49.750 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:49.747+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:49.770 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:49.768+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:50.730 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:50.728+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:50.776 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:50.774+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:51.767 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:51.764+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:51.804 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:51.802+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:52.777 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:52.775+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:52.786 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:52.783+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:53.777 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:53.775+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:53.799 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:53.797+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:54.754 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:54.752+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:54.790 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:54.788+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:55.711 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:55.708+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:55.771 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:55.769+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:56.676 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:56.674+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:56.725 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:56.722+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:57.646 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:57.644+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:57.746 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:57.743+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:58.657 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:58.654+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:58.720 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:58.717+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:59.655 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:12:59.652+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:12:59.683 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:12:59.681+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:00.644 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:00.642+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:00.654 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:00.652+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:01.604 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:01.601+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:01.701 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:01.698+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:02.596 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:02.594+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:02.682 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:02.679+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:03.574 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:03.572+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:03.698 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:03.696+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:04.538 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:04.536+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:04.654 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:04.651+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:05.537 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:05.535+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:05.623 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:05.621+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:06.561 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:06.558+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:06.642 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:06.640+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:07.536 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:07.534+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:07.612 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:07.610+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:08.577 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:08.574+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:08.585 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:08.583+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:09.569 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:09.566+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:09.598 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:09.596+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:10.568 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:10.566+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:10.596 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:10.593+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:11.539 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:11.536+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:11.592 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:11.590+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:12.496 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:12.493+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:12.571 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:12.551+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:13.533 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:13.530+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:13.539 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:13.537+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:14.523 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:14.522+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:14.530 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:14.527+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:15.521 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:15.519+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:15.573 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:15.571+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:16.546 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:16.544+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:16.569 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:16.568+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:17.526 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:17.524+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:17.588 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:17.586+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:18.567 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:18.566+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:18.570 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:18.569+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:19.531 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:19.529+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:19.568 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:19.567+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:20.497 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:20.495+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:20.561 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:20.559+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:21.486 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:21.484+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:21.531 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:21.529+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:22.520 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:22.519+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:22.527 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:22.526+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:23.489 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:23.488+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:23.515 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:23.513+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:24.486 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:24.485+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:24.487 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:24.486+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:25.481 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:25.480+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:25.482 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:25.480+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:26.451 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:26.449+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:26.518 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:26.516+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:27.420 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:27.418+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:27.533 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:27.531+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:28.438 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:28.437+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:28.503 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:28.502+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:29.455 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:29.453+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:29.484 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:29.483+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:30.461 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:30.460+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:30.502 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:30.500+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:31.435 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:31.434+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:31.462 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:31.460+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:32.443 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:32.441+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:32.473 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:32.472+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:33.426 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:33.424+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:33.482 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:33.479+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:34.437 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:34.435+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:34.441 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:34.439+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:35.416 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:35.413+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:35.452 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:35.449+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:36.435 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:36.433+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:36.478 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:36.475+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:37.399 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:37.397+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:37.517 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:37.514+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:38.384 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:38.381+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:38.472 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:38.469+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:39.378 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:39.376+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:39.445 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:39.443+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:40.415 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:40.413+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:40.456 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:40.454+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:41.449 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:41.446+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:41.495 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:41.493+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:42.455 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:42.452+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:42.489 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:42.487+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:43.412 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:43.409+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:43.475 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:43.473+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:44.446 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:44.444+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:44.509 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:44.507+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:45.483 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:45.481+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:45.490 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:45.488+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:46.446 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:46.443+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:46.522 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:46.520+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:47.431 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:47.429+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:47.519 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:47.516+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:48.399 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:48.397+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:48.479 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:48.477+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:49.412 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:49.409+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:49.449 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:49.447+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:50.387 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:50.384+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:50.490 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:50.487+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:51.381 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:51.379+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:51.477 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:51.475+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:52.365 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:52.363+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:52.523 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:52.520+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:53.415 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:53.413+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:53.491 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:53.489+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:54.404 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:54.402+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:54.532 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:54.529+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:55.392 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:55.389+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:55.526 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:55.524+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:56.363 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:56.360+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:56.569 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:56.565+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:57.345 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:57.342+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:57.591 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:57.588+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:58.301 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:58.298+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:58.618 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:58.615+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:59.269 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:13:59.267+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:13:59.600 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:13:59.597+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:00.228 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:00.225+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:00.600 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:00.598+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:01.187 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:01.184+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:01.603 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:01.601+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:02.231 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:02.229+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:02.648 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:02.645+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:03.253 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:03.251+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:03.652 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:03.649+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:04.292 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:04.290+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:04.644 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:04.642+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:05.250 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:05.248+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:05.687 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:05.685+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:06.277 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:06.274+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:06.718 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:06.716+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:07.327 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:07.325+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:07.679 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:07.677+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:08.338 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:08.336+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:08.637 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:08.635+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:09.387 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:09.385+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:09.646 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:09.644+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:10.341 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:10.339+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:10.625 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:10.624+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:11.353 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:11.352+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:11.664 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:11.663+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:12.373 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:12.371+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:12.681 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:12.680+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:13.371 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:13.369+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:13.665 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:13.663+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:14.399 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:14.397+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:14.645 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:14.644+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:15.447 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:15.446+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:15.650 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:15.648+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:16.421 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:16.420+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:16.696 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:16.694+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:17.447 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:17.446+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:17.654 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:17.653+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:18.476 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:18.474+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:18.642 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:18.640+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:19.465 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:19.464+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:19.648 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:19.647+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:20.487 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:20.485+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:20.647 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:20.646+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:21.464 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:21.462+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:21.680 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:21.679+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:22.415 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:22.414+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:22.640 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:22.638+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:23.379 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:23.378+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:23.680 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:23.678+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:24.337 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:24.336+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:24.645 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:24.643+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:25.321 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:25.319+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:25.607 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:25.604+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:26.319 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:26.316+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:26.634 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:26.632+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:27.321 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:27.318+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:27.641 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:27.638+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:28.315 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:28.312+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:28.625 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:28.623+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:29.313 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:29.310+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:29.636 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:29.633+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:30.325 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:30.323+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:30.644 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:30.641+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:31.339 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:31.336+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:31.680 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:31.678+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:32.366 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:32.363+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:32.699 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:32.697+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:33.333 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:33.330+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:33.742 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:33.739+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:34.318 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:34.316+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:34.719 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:34.717+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:35.288 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:35.286+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:35.689 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:35.687+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:36.322 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:36.319+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:36.669 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:36.666+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:37.304 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:37.301+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:37.695 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:37.693+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:38.290 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:38.287+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:38.714 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:38.711+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:39.305 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:39.303+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:39.764 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:39.762+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:40.287 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:40.285+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:40.768 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:40.766+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:41.269 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:41.267+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:41.720 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:41.718+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:42.248 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:42.245+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:42.719 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:42.717+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:43.200 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:43.198+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:43.711 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:43.708+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:44.200 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:44.197+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:44.740 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:44.737+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:45.193 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:45.190+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:45.783 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:45.781+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:46.148 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:46.146+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:46.768 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:46.765+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:47.100 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:47.097+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:47.796 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:47.794+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:48.055 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:48.053+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:48.821 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:48.819+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:49.103 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:49.101+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:49.895 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:49.847+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:50.092 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:50.090+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:50.833 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:50.830+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:51.101 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:51.098+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:51.821 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:51.819+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:52.137 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:52.135+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:52.845 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:52.843+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:53.161 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:53.159+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:53.830 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:53.827+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:54.139 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:54.137+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:54.833 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:54.831+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:55.118 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:55.115+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:55.841 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:55.839+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:56.160 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:56.158+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:56.886 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:56.883+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:57.139 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:57.137+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:57.856 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:57.855+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:58.124 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:58.122+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:58.874 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:58.872+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:59.077 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:14:59.076+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:14:59.919 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:14:59.918+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:00.062 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:00.060+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:00.965 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:00.962+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:01.075 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:01.074+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:01.931 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:01.930+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:02.113 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:02.112+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:02.950 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:02.949+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:03.077 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:03.075+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:03.932 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:03.930+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:04.097 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:04.096+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:04.930 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:04.928+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:05.125 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:05.124+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:05.914 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:05.912+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:06.157 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:06.155+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:06.944 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:06.943+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:07.138 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:07.137+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:07.963 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:07.962+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:08.172 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:08.170+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:08.990 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:08.988+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:09.154 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:09.153+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:09.977 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:09.975+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:10.135 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:10.134+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:10.975 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:10.974+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:11.152 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:11.151+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:12.015 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:12.014+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:12.176 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:12.174+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:12.981 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:12.979+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:13.225 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:13.223+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:13.933 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:13.931+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:14.232 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:14.231+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:14.942 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:14.941+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:15.241 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:15.239+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:15.929 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:15.926+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:16.212 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:16.211+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:16.945 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:16.943+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:17.261 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:17.258+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:17.961 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:17.959+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:18.309 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:18.307+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:18.918 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:18.915+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:19.295 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:19.293+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:19.945 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:19.943+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:20.321 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:20.319+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:20.906 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:20.903+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:21.329 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:21.327+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:21.929 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:21.927+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:22.354 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:22.351+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:22.888 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:22.885+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:23.305 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:23.302+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:23.929 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:23.926+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:24.298 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:24.296+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:24.917 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:24.915+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:25.340 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:25.337+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:25.880 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:25.877+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:26.362 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:26.359+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:26.861 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:26.858+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:27.407 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:27.404+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:27.851 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:27.848+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:28.435 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:28.433+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:28.844 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:28.842+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:29.406 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:29.404+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:29.889 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:29.887+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:30.447 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:30.445+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:30.908 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:30.906+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:31.451 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:31.449+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:31.865 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:31.862+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:32.449 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:32.446+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:32.876 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:32.873+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:33.411 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:33.408+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:33.864 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:33.861+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:34.374 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:34.371+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:34.853 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:34.851+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:35.395 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:35.392+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:35.885 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:35.883+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:36.418 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:36.416+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:36.932 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:36.930+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:37.400 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:37.397+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:37.960 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:37.957+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:38.446 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:38.443+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:38.921 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:38.919+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:39.426 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:39.424+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:39.927 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:39.924+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:40.438 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:40.436+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:40.974 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:40.972+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:41.460 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:41.457+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:41.990 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:41.987+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:42.444 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:42.442+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:42.957 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:42.954+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:43.459 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:43.456+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:43.966 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:43.963+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:44.446 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:44.444+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:45.015 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:45.013+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:45.473 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:45.471+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:45.974 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:45.972+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:46.426 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:46.424+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:47.022 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:47.020+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:47.389 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:47.387+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:48.029 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:48.027+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:48.361 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:48.359+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:49.008 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:49.005+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:49.350 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:49.348+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:50.000 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:49.998+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:50.316 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:50.313+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:51.038 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:51.036+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:51.353 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:51.352+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:52.016 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:52.015+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:52.365 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:52.364+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:52.981 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:52.980+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:53.335 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:53.333+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:53.967 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:53.966+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:54.351 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:54.349+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:54.998 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:54.996+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:55.338 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:55.337+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:56.018 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:56.017+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:56.384 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:56.383+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:57.037 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:57.034+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:57.358 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:57.357+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:58.075 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:58.074+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:58.338 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:58.336+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:59.110 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:15:59.109+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:15:59.356 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:15:59.355+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:00.111 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:00.109+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:00.348 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:00.347+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:01.142 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:01.141+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:01.387 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:01.385+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:02.180 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:02.178+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:02.425 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:02.423+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:03.166 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:03.165+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:03.417 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:03.415+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:04.146 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:04.145+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:04.446 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:04.445+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:05.154 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:05.153+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:05.458 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:05.456+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:06.117 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:06.115+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:06.489 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:06.487+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:07.107 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:07.105+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:07.464 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:07.463+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:08.074 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:08.072+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:08.436 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:08.434+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:09.123 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:09.121+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:09.455 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:09.452+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:10.138 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:10.136+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:10.439 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:10.437+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:11.155 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:11.152+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:11.459 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:11.456+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:12.132 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:12.129+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:12.480 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:12.478+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:13.101 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:13.099+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:13.490 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:13.487+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:14.088 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:14.085+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:14.454 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:14.452+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:15.138 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:15.135+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:15.407 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:15.404+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:16.095 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:16.092+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:16.432 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:16.430+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:17.135 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:17.132+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:17.474 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:17.472+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:18.147 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:18.145+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:18.495 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:18.493+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:19.144 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:19.142+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:19.473 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:19.471+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:20.116 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:20.114+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:20.456 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:20.453+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:21.150 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:21.148+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:21.475 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:21.473+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:22.155 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:22.153+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:22.470 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:22.468+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:23.128 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:23.125+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:23.499 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:23.497+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:24.102 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:24.100+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:24.510 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:24.507+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:25.130 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:25.127+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:25.553 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:25.551+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:26.150 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:26.147+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:26.520 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:26.518+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:27.107 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:27.105+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:27.566 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:27.564+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:28.114 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:28.111+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:28.565 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:28.563+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:29.094 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:29.091+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:29.577 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:29.575+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:30.104 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:30.102+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:30.566 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:30.563+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:31.137 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:31.134+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:31.590 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:31.587+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:32.122 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:32.120+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:32.627 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:32.624+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:33.098 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:33.096+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:33.626 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:33.623+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:34.084 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:34.082+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:34.660 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:34.658+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:35.100 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:35.097+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:35.634 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:35.632+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:36.064 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:36.062+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:36.609 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:36.606+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:37.092 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:37.090+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:37.655 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:37.652+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:38.069 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:38.066+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:38.644 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:38.642+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:39.036 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:39.034+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:39.605 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:39.603+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:40.083 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:40.081+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:40.571 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:40.569+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:41.119 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:41.117+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:41.607 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:41.606+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:42.115 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:42.112+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:42.640 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:42.638+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:43.131 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:43.130+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:43.603 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:43.601+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:44.110 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:44.108+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:44.623 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:44.621+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:45.100 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:45.098+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:45.585 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:45.584+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:46.113 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:46.111+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:46.565 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:46.564+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:47.131 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:47.129+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:47.564 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:47.563+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:48.089 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:48.088+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:48.600 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:48.599+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:49.111 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:49.110+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:49.561 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:49.559+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:50.103 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:50.101+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:50.517 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:50.516+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:51.101 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:51.100+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:51.538 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:51.537+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:52.101 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:52.100+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:52.533 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:52.531+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:53.070 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:53.069+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:53.573 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:53.572+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:54.065 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:54.064+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:54.610 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:54.609+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:55.085 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:55.083+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:55.561 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:55.560+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:56.050 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:56.049+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:56.577 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:56.576+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:57.093 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:57.092+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:57.529 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:57.528+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:58.065 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:58.064+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:58.525 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:58.524+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:59.097 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:16:59.096+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:16:59.533 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:16:59.530+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:00.115 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:00.113+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:00.557 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:00.555+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:01.073 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:01.070+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:01.537 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:01.534+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:02.075 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:02.073+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:02.545 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:02.543+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:03.085 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:03.082+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:03.580 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:03.577+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:04.107 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:04.104+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:04.630 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:04.627+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:05.063 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:05.061+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:05.642 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:05.639+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:06.098 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:06.095+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:06.671 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:06.669+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:07.147 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:07.145+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:07.649 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:07.647+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:08.150 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:08.147+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:08.674 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:08.671+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:09.191 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:09.189+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:09.715 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:09.712+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:10.176 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:10.173+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:10.684 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:10.682+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:11.147 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:11.145+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:11.669 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:11.667+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:12.176 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:12.173+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:12.653 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:12.651+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:13.138 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:13.135+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:13.673 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:13.670+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:14.134 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:14.132+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:14.692 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:14.690+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:15.094 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:15.092+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:15.672 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:15.669+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:16.126 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:16.123+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:16.697 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:16.695+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:17.097 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:17.095+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:17.735 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:17.733+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:18.097 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:18.095+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:18.718 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:18.716+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:19.060 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:19.058+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:19.670 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:19.667+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:20.022 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:20.020+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:20.645 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:20.642+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:21.021 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:21.018+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:21.602 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:21.600+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:22.032 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:22.030+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:22.621 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:22.618+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:23.042 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:23.039+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:23.623 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:23.621+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:24.005 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:24.001+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:24.626 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:24.624+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:25.045 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:25.043+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:25.675 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:25.672+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:26.003 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:26.000+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:26.629 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:26.626+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:26.991 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:26.989+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:27.636 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:27.634+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:27.998 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:27.996+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:28.662 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:28.659+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:29.018 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:29.014+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:29.663 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:29.660+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:30.041 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:30.039+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:30.627 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:30.624+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:30.992 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:30.990+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:31.665 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:31.663+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:32.030 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:32.027+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:32.635 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:32.634+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:33.075 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:33.074+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:33.681 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:33.680+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:34.052 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:34.051+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:34.645 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:34.643+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:35.087 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:35.085+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:35.656 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:35.655+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:36.125 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:36.123+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:36.700 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:36.699+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:37.108 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:37.107+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:37.697 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:37.696+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:38.148 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:38.147+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:38.694 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:38.693+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:39.146 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:39.144+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:39.655 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:39.654+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:40.132 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:40.130+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:40.653 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:40.652+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:41.152 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:41.149+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:41.642 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:41.641+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:42.150 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:42.149+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:42.596 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:42.595+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:43.172 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:43.171+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:43.615 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:43.614+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:44.161 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:44.159+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:44.646 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:44.644+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:45.145 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:45.143+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:45.641 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:45.637+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:46.099 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:46.097+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:46.594 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:46.593+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:47.104 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:47.102+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:47.547 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:47.546+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:48.116 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:48.114+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:48.518 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:48.517+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:49.134 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:49.133+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:49.584 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:49.524+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:50.121 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:50.120+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:50.527 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:50.525+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:51.115 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:51.113+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:51.486 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:51.483+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:52.166 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:52.164+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:52.478 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:52.476+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:53.183 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:53.181+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:53.490 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:53.488+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:54.215 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:54.212+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:54.498 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:54.496+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:55.218 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:55.216+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:55.500 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:55.497+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:56.211 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:56.208+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:56.522 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:56.519+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:57.227 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:57.225+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:57.504 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:57.502+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:58.230 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:58.227+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:58.508 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:58.506+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:59.203 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:17:59.201+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:17:59.530 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:17:59.527+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:00.182 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:00.179+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:00.518 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:00.515+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:01.170 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:01.168+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:01.538 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:01.536+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:02.184 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:02.182+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:02.567 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:02.564+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:03.174 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:03.172+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:03.526 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:03.524+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:04.133 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:04.130+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:04.494 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:04.491+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:05.176 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:05.174+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:05.499 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:05.497+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:06.205 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:06.202+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:06.530 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:06.527+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:07.202 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:07.200+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:07.517 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:07.514+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:08.185 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:08.183+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:08.525 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:08.523+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:09.173 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:09.171+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:09.486 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:09.484+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:10.219 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:10.217+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:10.485 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:10.482+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:11.209 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:11.206+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:11.458 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:11.456+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:12.253 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:12.251+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:12.493 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:12.490+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:13.207 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:13.205+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:13.510 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:13.508+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:14.222 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:14.219+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:14.520 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:14.517+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:15.195 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:15.193+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:15.498 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:15.495+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:16.165 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:16.163+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:16.452 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:16.450+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:17.142 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:17.139+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:17.408 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:17.406+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:18.099 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:18.097+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:18.390 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:18.387+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:19.120 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:19.118+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:19.405 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:19.402+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:20.152 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:20.150+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:20.383 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:20.381+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:21.152 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:21.149+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:21.373 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:21.370+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:22.145 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:22.142+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:22.363 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:22.360+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:23.176 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:23.173+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:23.341 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:23.339+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:24.176 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:24.175+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:24.319 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:24.318+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:25.217 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:25.216+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:25.359 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:25.358+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:26.211 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:26.209+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:26.364 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:26.362+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:27.166 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:27.164+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:27.321 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:27.319+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:28.142 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:28.140+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:28.301 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:28.300+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:29.173 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:29.172+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:29.347 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:29.345+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:30.165 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:30.163+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:30.334 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:30.333+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:31.188 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:31.187+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:31.292 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:31.291+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:32.197 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:32.193+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:32.289 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:32.287+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:33.168 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:33.167+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:33.248 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:33.246+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:34.138 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:34.137+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:34.262 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:34.261+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:35.142 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:35.140+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:35.246 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:35.245+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:36.158 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:36.157+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:36.273 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:36.271+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:37.125 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:37.123+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:37.258 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:37.257+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:38.157 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:38.155+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:38.277 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:38.275+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:39.193 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:39.192+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:39.305 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:39.303+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:40.197 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:40.196+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:40.282 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:40.281+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:41.186 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:41.185+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:41.258 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:41.256+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:42.166 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:42.164+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:42.271 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:42.269+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:43.206 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:43.204+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:43.316 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:43.314+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:44.190 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:44.188+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:44.280 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:44.278+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:45.145 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:45.143+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:45.328 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:45.326+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:46.183 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:46.180+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:46.317 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:46.315+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:47.134 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:47.132+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:47.302 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:47.299+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:48.088 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:48.085+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:48.347 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:48.344+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:49.083 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:49.081+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:49.320 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:49.317+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:50.046 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:50.044+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:50.356 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:50.354+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:51.076 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:51.073+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:51.361 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:51.358+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:52.092 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:52.089+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:52.380 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:52.378+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:53.078 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:53.076+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:53.415 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:53.413+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:54.120 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:54.117+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:54.406 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:54.403+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:55.135 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:55.133+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:55.391 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:55.389+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:56.142 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:56.140+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:56.417 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:56.415+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:57.143 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:57.141+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:57.428 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:57.426+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:58.193 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:58.191+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:58.432 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:58.430+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:59.224 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:18:59.222+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:18:59.418 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:18:59.415+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:00.245 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:00.243+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:00.379 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:00.377+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:01.210 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:01.208+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:01.340 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:01.338+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:02.168 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:02.166+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:02.313 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:02.309+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:03.123 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:03.121+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:03.342 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:03.340+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:04.157 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:04.154+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:04.383 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:04.381+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:05.173 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:05.170+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:05.393 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:05.390+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:06.127 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:06.124+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:06.358 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:06.355+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:07.137 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:07.135+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:07.350 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:07.347+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:08.113 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:08.111+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:08.356 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:08.353+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:09.114 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:09.111+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:09.401 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:09.398+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:10.075 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:10.073+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:10.450 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:10.448+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:11.071 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:11.069+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:11.466 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:11.463+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:12.046 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:12.044+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:12.423 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:12.421+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:13.091 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:13.088+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:13.393 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:13.391+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:14.060 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:14.058+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:14.380 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:14.378+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:15.032 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:15.030+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:15.429 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:15.428+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:16.046 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:16.045+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:16.389 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:16.387+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:17.071 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:17.070+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:17.381 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:17.380+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:18.032 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:18.030+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:18.407 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:18.406+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:19.027 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:19.025+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:19.365 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:19.364+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:20.028 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:20.026+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:20.328 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:20.327+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:20.983 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:20.981+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:21.304 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:21.303+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:21.971 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:21.970+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:22.313 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:22.312+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:22.967 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:22.965+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:23.359 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:23.358+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:23.927 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:23.926+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:24.398 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:24.396+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:24.943 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:24.942+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:25.400 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:25.399+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:25.952 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:25.950+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:26.429 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:26.427+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:26.962 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:26.960+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:27.480 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:27.478+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:27.972 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:27.970+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:28.479 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:28.478+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:28.952 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:28.951+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:29.490 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:29.488+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:30.003 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:30.002+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:30.538 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:30.536+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:30.973 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:30.972+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:31.542 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:31.540+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:31.963 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:31.961+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:32.495 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:32.493+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:32.989 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:32.986+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:33.493 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:33.491+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:33.969 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:33.967+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:34.488 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:34.485+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:34.990 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:34.987+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:35.487 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:35.484+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:36.037 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:36.035+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:36.450 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:36.448+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:36.991 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:36.989+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:37.436 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:37.433+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:37.953 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:37.951+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:38.425 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:38.422+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:38.927 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:38.924+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:39.416 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:39.414+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:39.913 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:39.911+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:40.410 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:40.407+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:40.959 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:40.957+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:41.389 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:41.387+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:41.913 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:41.910+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:42.394 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:42.391+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:42.911 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:42.909+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:43.357 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:43.355+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:43.877 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:43.875+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:44.350 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:44.347+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:44.894 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:44.891+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:45.317 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:45.315+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:45.849 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:45.847+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:46.314 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:46.312+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:46.896 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:46.894+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:47.296 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:47.293+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:47.873 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:47.871+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:48.317 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:48.314+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:48.885 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:48.883+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:49.367 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:49.365+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:49.911 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:49.908+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:50.328 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:50.325+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:50.891 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:50.889+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:51.304 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:51.302+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:51.913 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:51.911+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:52.348 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:52.345+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:52.938 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:52.935+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:53.299 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:53.297+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:53.919 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:53.916+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:54.328 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:54.326+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:54.911 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:54.908+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:55.299 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:55.296+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:55.942 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:55.940+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:56.337 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:56.334+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:56.929 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:56.926+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:57.330 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:57.328+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:57.886 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:57.883+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:58.348 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:58.345+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:58.843 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:58.841+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:59.344 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:19:59.341+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:19:59.869 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:19:59.867+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:00.367 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:00.364+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:00.863 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:00.861+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:01.390 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:01.387+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:01.839 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:01.836+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:02.373 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:02.370+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:02.819 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:02.817+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:03.420 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:03.418+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:03.816 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:03.814+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:04.406 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:04.404+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:04.799 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:04.797+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:05.376 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:05.374+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:05.849 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:05.848+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:06.388 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:06.387+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:06.856 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:06.854+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:07.434 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:07.433+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:07.842 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:07.841+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:08.453 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:08.451+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:08.874 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:08.873+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:09.428 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:09.427+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:09.830 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:09.829+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:10.433 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:10.431+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:10.850 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:10.849+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:11.458 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:11.456+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:11.860 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:11.859+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:12.454 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:12.453+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:12.900 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:12.898+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:13.429 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:13.427+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:13.904 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:13.902+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:14.450 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:14.449+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:14.860 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:14.858+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:15.413 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:15.412+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:15.894 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:15.893+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:16.418 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:16.417+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:16.859 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:16.858+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:17.400 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:17.399+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:17.826 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:17.825+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:18.410 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:18.409+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:18.869 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:18.868+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:19.415 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:19.414+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:19.878 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:19.877+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:20.448 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:20.447+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:20.889 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:20.888+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:21.407 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:21.405+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:21.937 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:21.936+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:22.402 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:22.400+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:22.979 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:22.976+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:23.370 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:23.368+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:24.027 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:24.025+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:24.337 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:24.335+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:25.050 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:25.047+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:25.351 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:25.348+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:26.055 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:26.052+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:26.371 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:26.368+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:27.030 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:27.028+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:27.332 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:27.330+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:27.999 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:27.996+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:28.367 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:28.365+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:29.006 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:29.003+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:29.398 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:29.396+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:29.965 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:29.963+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:30.442 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:30.440+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:30.960 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:30.958+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:31.465 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:31.463+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:31.994 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:31.992+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:32.484 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:32.482+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:32.963 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:32.961+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:33.498 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:33.496+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:33.939 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:33.937+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:34.460 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:34.457+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:34.962 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:34.959+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:35.486 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:35.483+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:35.946 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:35.943+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:36.442 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:36.439+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:36.982 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:36.979+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:37.432 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:37.429+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:37.955 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:37.953+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:38.428 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:38.426+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:38.946 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:38.943+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:39.462 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:39.460+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:39.982 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:39.980+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:40.485 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:40.482+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:40.975 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:40.972+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:41.511 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:41.508+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:41.986 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:41.984+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:42.528 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:42.525+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:43.003 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:43.000+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:43.544 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:43.541+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:44.049 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:44.047+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:44.572 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:44.569+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:45.058 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:45.056+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:45.618 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:45.615+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:46.106 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:46.104+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:46.625 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:46.622+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:47.146 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:47.143+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:47.600 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:47.598+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:48.097 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:48.095+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:48.649 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:48.645+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:49.098 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:49.096+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:49.641 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:49.638+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:50.071 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:50.068+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:50.609 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:50.606+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:51.085 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:51.082+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:51.573 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:51.571+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:52.061 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:52.059+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:52.544 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:52.542+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:53.069 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:53.067+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:53.516 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:53.514+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:54.080 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:54.078+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:54.482 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:54.480+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:55.102 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:55.100+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:55.516 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:55.514+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:56.125 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:56.124+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:56.472 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:56.470+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:57.151 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:57.150+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:57.515 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:57.514+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:58.171 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:58.169+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:58.531 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:58.530+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:59.172 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:20:59.171+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:20:59.514 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:20:59.513+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:00.133 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:00.131+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:00.558 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:00.557+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:01.152 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:01.150+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:01.513 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:01.512+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:02.135 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:02.133+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:02.525 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:02.524+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:03.122 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:03.121+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:03.522 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:03.519+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:04.109 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:04.108+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:04.519 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:04.518+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:05.102 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:05.101+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:05.482 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:05.480+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:06.131 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:06.130+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:06.511 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:06.510+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:07.110 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:07.109+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:07.471 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:07.470+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:08.115 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:08.114+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:08.492 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:08.490+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:09.126 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:09.124+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:09.472 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:09.470+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:10.099 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:10.098+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:10.493 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:10.491+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:11.118 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:11.116+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:11.453 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:11.452+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:12.111 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:12.110+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:12.415 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:12.413+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:13.122 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:13.121+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:13.465 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:13.463+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:14.094 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:14.091+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:14.431 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:14.429+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:15.070 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:15.068+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:15.457 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:15.455+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:16.066 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:16.064+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:16.479 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:16.477+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:17.017 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:17.014+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:17.508 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:17.506+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:17.999 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:17.996+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:18.520 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:18.517+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:18.987 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:18.985+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:19.571 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:19.568+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:19.988 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:19.985+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:20.598 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:20.596+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:20.957 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:20.955+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:21.596 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:21.593+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:21.966 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:21.964+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:22.633 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:22.631+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:22.998 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:22.996+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:23.607 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:23.605+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:24.045 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:24.043+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:24.626 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:24.624+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:25.008 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:25.005+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:25.670 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:25.668+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:26.057 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:26.055+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:26.661 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:26.658+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:27.084 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:27.081+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:27.639 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:27.636+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:28.055 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:28.053+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:28.622 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:28.619+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:29.093 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:29.091+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:29.664 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:29.662+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:30.139 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:30.137+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:30.698 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:30.695+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:31.173 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:31.170+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:31.693 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:31.691+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:32.182 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:32.179+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:32.677 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:32.674+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:33.165 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:33.162+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:33.691 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:33.688+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:34.213 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:34.211+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:34.654 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:34.652+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:35.244 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:35.242+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:35.645 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:35.643+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:36.252 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:36.249+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:36.676 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:36.674+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:37.215 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:37.212+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:37.687 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:37.685+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:38.209 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:38.206+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:38.736 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:38.733+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:39.190 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:39.188+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:39.761 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:39.759+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:40.210 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:40.207+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:40.758 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:40.756+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:41.253 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:41.251+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:41.769 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:41.767+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:42.259 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:42.257+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:42.799 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:42.797+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:43.289 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:43.287+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:43.841 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:43.838+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:44.253 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:44.251+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:44.869 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:44.867+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:45.263 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:45.261+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:45.874 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:45.872+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:46.297 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:46.295+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:46.871 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:46.869+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:47.331 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:47.330+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:47.848 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:47.846+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:48.329 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:48.328+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:48.831 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:48.830+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:49.319 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:49.318+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:49.787 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:49.786+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:50.273 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:50.272+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:50.794 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:50.793+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:51.247 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:51.246+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:51.825 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:51.824+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:52.250 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:52.248+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:52.797 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:52.796+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:53.260 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:53.259+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:53.758 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:53.756+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:54.238 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:54.237+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:54.803 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:54.801+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:55.209 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:55.208+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:55.762 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:55.761+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:56.247 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:56.246+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:56.772 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:56.770+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:57.230 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:57.229+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:57.781 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:57.780+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:58.225 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:58.224+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:58.809 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:58.808+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:59.196 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:21:59.195+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:21:59.840 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:21:59.839+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:00.181 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:00.180+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:00.805 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:00.804+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:01.143 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:01.142+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:01.790 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:01.788+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:02.175 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:02.173+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:02.784 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:02.783+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:03.186 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:03.185+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:03.812 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:03.810+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:04.229 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:04.228+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:04.777 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:04.774+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:05.249 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:05.247+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:05.753 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:05.750+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:06.274 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:06.272+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:06.796 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:06.794+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:07.267 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:07.265+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:07.767 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:07.764+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:08.267 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:08.265+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:08.794 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:08.791+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:09.243 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:09.240+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:09.823 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:09.821+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:10.293 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:10.291+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:10.803 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:10.801+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:11.248 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:11.246+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:11.772 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:11.770+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:12.213 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:12.210+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:12.794 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:12.792+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:13.206 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:13.203+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:13.831 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:13.828+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:14.166 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:14.164+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:14.828 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:14.825+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:15.204 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:15.201+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:15.791 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:15.789+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:16.191 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:16.188+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:16.753 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:16.750+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:17.238 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:17.236+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:17.708 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:17.705+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:18.216 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:18.213+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:18.681 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:18.678+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:19.249 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:19.246+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:19.678 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:19.676+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:20.275 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:20.273+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:20.643 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:20.641+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:21.320 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:21.317+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:21.645 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:21.643+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:22.276 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:22.274+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:22.683 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:22.680+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:23.321 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:23.318+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:23.667 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:23.664+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:24.368 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:24.365+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:24.619 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:24.617+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:25.405 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:25.402+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:25.662 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:25.659+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:26.381 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:26.378+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:26.624 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:26.622+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:27.343 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:27.340+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:27.578 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:27.576+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:28.294 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:28.291+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:28.546 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:28.543+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:29.269 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:29.267+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:29.567 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:29.565+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:30.277 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:30.274+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:30.591 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:30.589+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:31.259 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:31.256+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:31.549 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:31.547+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:32.265 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:32.262+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:32.501 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:32.499+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:33.287 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:33.285+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:33.499 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:33.497+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:34.295 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:34.292+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:34.523 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:34.520+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:35.293 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:35.291+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:35.487 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:35.484+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:36.296 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:36.294+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:36.460 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:36.458+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:37.321 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:37.319+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:37.428 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:37.426+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:38.286 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:38.285+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:38.426 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:38.425+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:39.312 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:39.310+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:39.419 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:39.418+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:40.309 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:40.308+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:40.438 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:40.437+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:41.338 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:41.336+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:41.475 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:41.473+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:42.381 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:42.379+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:42.468 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:42.467+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:43.342 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:43.341+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:43.435 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:43.434+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:44.358 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:44.356+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:44.417 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:44.415+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:45.314 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:45.312+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:45.420 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:45.419+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:46.317 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:46.315+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:46.378 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:46.377+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:47.286 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:47.285+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:47.381 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:47.379+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:48.333 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:48.331+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:48.359 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:48.358+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:49.374 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:49.372+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:49.400 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:49.399+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:50.406 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:50.405+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:50.418 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:50.417+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:51.394 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:51.393+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:51.432 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:51.430+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:52.391 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:52.390+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:52.414 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:52.412+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:53.406 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:53.405+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:53.429 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:53.428+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:54.427 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:54.426+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:54.443 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:54.441+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:55.451 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:55.450+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:55.475 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:55.473+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:56.473 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:56.471+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:56.485 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:56.482+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:57.486 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:57.484+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:57.505 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:57.503+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:58.473 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:58.471+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:58.477 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:58.475+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:59.470 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:22:59.467+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:22:59.474 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:22:59.472+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:00.470 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:00.468+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:00.520 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:00.517+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:01.502 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:01.500+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:01.514 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:01.512+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:02.496 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:02.494+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:02.503 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:02.501+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:03.484 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:03.481+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:03.530 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:03.528+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:04.440 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:04.438+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:04.484 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:04.482+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:05.433 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:05.430+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:05.494 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:05.491+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:06.416 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:06.414+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:06.502 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:06.499+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:07.404 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:07.401+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:07.482 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:07.480+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:08.357 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:08.355+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:08.448 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:08.446+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:09.313 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:09.311+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:09.434 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:09.431+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:10.310 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:10.307+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:10.476 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:10.473+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:11.271 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:11.269+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:11.454 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:11.451+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:12.263 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:12.261+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:12.406 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:12.403+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:13.219 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:13.217+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:13.422 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:13.419+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:14.189 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:14.187+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:14.396 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:14.394+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:15.193 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:15.190+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:15.375 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:15.373+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:16.223 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:16.220+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:16.331 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:16.329+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:17.188 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:17.185+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:17.346 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:17.344+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:18.230 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:18.228+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:18.345 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:18.342+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:19.268 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:19.265+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:19.374 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:19.371+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:20.223 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:20.220+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:20.342 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:20.339+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:21.218 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:21.216+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:21.379 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:21.377+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:22.213 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:22.211+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:22.346 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:22.344+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:23.222 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:23.219+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:23.306 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:23.303+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:24.268 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:24.266+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:24.290 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:24.287+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:25.237 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:25.235+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:25.317 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:25.314+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:26.249 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:26.246+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:26.284 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:26.282+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:27.283 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:27.281+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:27.302 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:27.296+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:28.266 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:28.264+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:28.327 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:28.325+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:29.243 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:29.242+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:29.278 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:29.276+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:30.283 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:30.282+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:30.318 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:30.317+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:31.239 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:31.238+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:31.347 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:31.345+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:32.282 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:32.281+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:32.349 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:32.348+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:33.329 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:33.327+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:33.346 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:33.345+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:34.336 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:34.334+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:34.341 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:34.340+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:35.317 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:35.316+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:35.328 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:35.327+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:36.351 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:36.349+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:36.358 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:36.357+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:37.370 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:37.369+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:37.390 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:37.389+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:38.366 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:38.365+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:38.397 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:38.396+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:39.389 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:39.388+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:39.395 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:39.394+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:40.386 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:40.384+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:40.402 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:40.401+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:41.373 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:41.372+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:41.425 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:41.424+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:42.339 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:42.338+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:42.386 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:42.384+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:43.378 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:43.376+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:43.407 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:43.405+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:44.395 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:44.393+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:44.420 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:44.419+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:45.377 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:45.376+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:45.469 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:45.467+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:46.419 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:46.417+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:46.430 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:46.429+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:47.467 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:47.464+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:47.467 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:47.465+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:48.426 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:48.424+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:48.486 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:48.483+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:49.439 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:49.437+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:49.472 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:49.470+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:50.407 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:50.405+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:50.455 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:50.453+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:51.381 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:51.378+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:51.479 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:51.477+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:52.381 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:52.379+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:52.446 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:52.444+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:53.341 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:53.339+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:53.399 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:53.397+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:54.299 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:54.296+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:54.437 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:54.435+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:55.290 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:55.288+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:55.413 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:55.410+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:56.285 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:56.283+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:56.390 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:56.388+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:57.273 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:57.270+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:57.430 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:57.427+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:58.259 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:58.256+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:58.477 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:58.475+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:59.241 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:23:59.238+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:23:59.501 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:23:59.498+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:00.254 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:00.251+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:00.534 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:00.532+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:01.213 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:01.210+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:01.535 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:01.533+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:02.205 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:02.202+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:02.581 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:02.579+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:03.212 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:03.210+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:03.573 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:03.571+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:04.234 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:04.232+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:04.566 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:04.564+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:05.276 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:05.273+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:05.581 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:05.579+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:06.258 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:06.256+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:06.619 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:06.617+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:07.251 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:07.249+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:07.573 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:07.571+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:08.275 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:08.273+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:08.545 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:08.543+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:09.243 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:09.241+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:09.543 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:09.541+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:10.233 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:10.230+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:10.563 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:10.560+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:11.284 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:11.281+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:11.593 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:11.590+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:12.330 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:12.328+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:12.636 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:12.634+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:13.340 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:13.338+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:13.677 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:13.675+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:14.299 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:14.296+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:14.714 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:14.712+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:15.349 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:15.346+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:15.711 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:15.709+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:16.367 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:16.365+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:16.740 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:16.737+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:17.370 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:17.368+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:17.772 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:17.770+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:18.332 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:18.330+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:18.764 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:18.762+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:19.379 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:19.376+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:19.805 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:19.803+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:20.401 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:20.399+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:20.772 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:20.771+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:21.375 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:21.374+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:21.818 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:21.817+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:22.339 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:22.337+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:22.787 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:22.786+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:23.308 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:23.307+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:23.800 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:23.799+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:24.317 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:24.315+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:24.808 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:24.806+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:25.302 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:25.301+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:25.794 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:25.793+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:26.273 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:26.272+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:26.816 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:26.815+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:27.292 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:27.291+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:27.860 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:27.858+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:28.291 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:28.289+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:28.893 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:28.891+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:29.274 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:29.273+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:29.852 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:29.850+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:30.255 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:30.254+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:30.879 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:30.878+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:31.254 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:31.252+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:31.856 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:31.855+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:32.240 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:32.238+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:32.892 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:32.890+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:33.214 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:33.213+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:33.899 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:33.898+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:34.165 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:34.164+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:34.922 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:34.920+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:35.132 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:35.129+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:35.932 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:35.931+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:36.148 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:36.147+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:36.914 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:36.913+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:37.167 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:37.166+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:37.881 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:37.880+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:38.166 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:38.163+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:38.904 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:38.902+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:39.144 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:39.142+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:39.929 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:39.927+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:40.136 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:40.134+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:40.905 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:40.903+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:41.156 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:41.153+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:41.910 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:41.908+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:42.202 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:42.200+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:42.873 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:42.870+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:43.193 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:43.191+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:43.885 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:43.882+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:44.222 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:44.219+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:44.856 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:44.854+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:45.216 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:45.214+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:45.850 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:45.847+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:46.184 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:46.182+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:46.807 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:46.805+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:47.177 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:47.174+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:47.805 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:47.803+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:48.185 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:48.183+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:48.786 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:48.783+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:49.196 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:49.194+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:49.797 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:49.794+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:50.230 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:50.228+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:50.773 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:50.771+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:51.227 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:51.224+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:51.734 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:51.732+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:52.206 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:52.203+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:52.734 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:52.731+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:53.176 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:53.174+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:53.721 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:53.718+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:54.191 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:54.188+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:54.743 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:54.741+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:55.226 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:55.223+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:55.716 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:55.713+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:56.268 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:56.265+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:56.763 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:56.761+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:57.300 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:57.298+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:57.805 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:57.803+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:58.321 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:58.319+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:58.847 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:58.845+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:59.292 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:24:59.289+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:24:59.866 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:24:59.863+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:00.334 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:00.332+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:00.893 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:00.891+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:01.345 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:01.342+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:01.847 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:01.845+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:02.361 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:02.359+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:02.892 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:02.889+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:03.325 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:03.323+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:03.874 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:03.871+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:04.310 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:04.308+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:04.907 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:04.904+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:05.294 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:05.291+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:05.894 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:05.891+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:06.311 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:06.308+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:06.870 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:06.868+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:07.343 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:07.340+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:07.851 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:07.849+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:08.342 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:08.340+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:08.824 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:08.822+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:09.385 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:09.383+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:09.780 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:09.778+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:10.437 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:10.432+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:10.754 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:10.751+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:11.448 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:11.446+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:11.758 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:11.756+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:12.482 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:12.480+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:12.719 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:12.718+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:13.488 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:13.487+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:13.706 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:13.705+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:14.528 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:14.527+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:14.710 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:14.709+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:15.504 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:15.503+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:15.752 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:15.751+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:16.485 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:16.484+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:16.788 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:16.787+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:17.471 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:17.469+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:17.772 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:17.770+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:18.500 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:18.498+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:18.782 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:18.781+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:19.498 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:19.497+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:19.760 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:19.759+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:20.518 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:20.516+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:20.759 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:20.758+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:21.507 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:21.505+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:21.741 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:21.739+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:22.542 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:22.540+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:22.766 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:22.765+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:23.540 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:23.539+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:23.748 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:23.747+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:24.500 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:24.499+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:24.743 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:24.742+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:25.473 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:25.471+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:25.715 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:25.712+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:26.510 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:26.509+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:26.751 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:26.749+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:27.535 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:27.533+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:27.733 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:27.732+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:28.550 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:28.549+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:28.690 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:28.689+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:29.570 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:29.568+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:29.728 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:29.726+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:30.548 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:30.545+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:30.770 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:30.768+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:31.504 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:31.502+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:31.748 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:31.746+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:32.550 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:32.547+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:32.699 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:32.696+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:33.597 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:33.595+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:33.720 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:33.718+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:34.627 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:34.625+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:34.740 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:34.738+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:35.604 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:35.602+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:35.767 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:35.765+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:36.625 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:36.622+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:36.737 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:36.734+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:37.634 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:37.632+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:37.697 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:37.695+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:38.658 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:38.655+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:38.739 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:38.736+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:39.676 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:39.673+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:39.787 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:39.785+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:40.648 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:40.646+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:40.770 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:40.768+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:41.603 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:41.601+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:41.749 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:41.747+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:42.622 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:42.619+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:42.798 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:42.795+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:43.672 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:43.670+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:43.749 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:43.747+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:44.630 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:44.628+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:44.766 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:44.764+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:45.660 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:45.658+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:45.740 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:45.738+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:46.618 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:46.615+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:46.706 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:46.704+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:47.613 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:47.611+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:47.743 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:47.740+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:48.640 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:48.638+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:48.741 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:48.738+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:49.624 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:49.621+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:49.752 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:49.750+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:50.602 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:50.599+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:50.722 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:50.720+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:51.618 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:51.616+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:51.684 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:51.682+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:52.646 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:52.644+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:52.698 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:52.695+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:53.628 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:53.626+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:53.698 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:53.694+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:54.658 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:54.655+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:54.745 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:54.743+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:55.627 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:55.625+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:55.709 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:55.707+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:56.600 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:56.598+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:56.670 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:56.668+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:57.639 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:57.636+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:57.718 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:57.715+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:58.626 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:58.624+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:58.766 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:58.764+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:59.662 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:25:59.660+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:25:59.732 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:25:59.730+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:00.691 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:00.688+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:00.768 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:00.765+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:01.719 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:01.717+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:01.807 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:01.805+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:02.733 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:02.731+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:02.822 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:02.821+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:03.775 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:03.773+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:03.849 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:03.848+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:04.767 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:04.766+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:04.893 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:04.892+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:05.736 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:05.735+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:05.923 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:05.922+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:06.718 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:06.716+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:06.945 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:06.944+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:07.764 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:07.762+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:07.936 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:07.934+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:08.731 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:08.730+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:08.943 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:08.942+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:09.770 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:09.769+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:09.984 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:09.982+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:10.770 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:10.768+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:10.957 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:10.956+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:11.799 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:11.798+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:12.000 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:11.999+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:12.848 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:12.846+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:13.022 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:13.021+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:13.825 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:13.823+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:13.987 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:13.985+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:14.786 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:14.784+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:14.951 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:14.950+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:15.756 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:15.755+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:15.912 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:15.911+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:16.795 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:16.794+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:16.865 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:16.864+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:17.839 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:17.838+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:17.882 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:17.881+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:18.818 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:18.817+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:18.889 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:18.888+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:19.806 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:19.804+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:19.923 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:19.920+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:20.850 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:20.848+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:20.928 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:20.926+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:21.875 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:21.872+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:21.954 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:21.951+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:22.830 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:22.827+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:22.971 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:22.969+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:23.832 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:23.830+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:23.986 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:23.984+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:24.832 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:24.830+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:24.953 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:24.950+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:25.870 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:25.868+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:25.941 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:25.939+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:26.851 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:26.848+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:26.971 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:26.969+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:27.888 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:27.886+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:27.933 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:27.930+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:28.903 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:28.900+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:28.943 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:28.941+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:29.892 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:29.890+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:29.986 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:29.983+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:30.917 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:30.914+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:30.947 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:30.945+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:31.966 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:31.964+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:31.977 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:31.975+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:32.928 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:32.925+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:33.023 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:33.021+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:33.881 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:33.878+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:34.046 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:34.043+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:34.913 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:34.911+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:35.030 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:35.028+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:35.889 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:35.887+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:36.043 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:36.041+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:36.845 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:36.843+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:37.025 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:37.023+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:37.841 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:37.838+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:38.075 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:38.072+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:38.888 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:38.886+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:39.026 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:39.024+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:39.931 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:39.929+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:40.020 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:40.017+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:40.969 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:40.967+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:41.092 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:41.053+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:42.014 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:42.011+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:42.019 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:42.017+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:43.057 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:43.055+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:43.058 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:43.055+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:44.016 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:44.014+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:44.025 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:44.023+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:44.987 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:44.985+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:45.052 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:45.049+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:45.950 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:45.947+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:46.046 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:46.044+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:46.929 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:46.927+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:47.044 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:47.042+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:47.955 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:47.953+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:48.085 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:48.083+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:48.919 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:48.916+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:49.067 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:49.065+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:49.962 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:49.960+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:50.072 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:50.070+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:50.992 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:50.990+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:51.070 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:51.068+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:51.987 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:51.985+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:52.056 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:52.054+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:53.010 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:53.009+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:53.102 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:53.100+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:54.038 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:54.037+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:54.129 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:54.128+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:54.996 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:54.995+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:55.100 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:55.098+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:55.970 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:55.969+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:56.064 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:56.063+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:57.002 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:57.000+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:57.024 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:57.023+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:57.975 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:57.972+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:57.997 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:57.995+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:58.984 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:58.982+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:59.001 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:59.000+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:26:59.941 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:26:59.939+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:00.000 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:26:59.998+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:00.934 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:00.933+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:01.042 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:01.040+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:01.913 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:01.910+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:02.058 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:02.057+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:02.903 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:02.902+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:03.052 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:03.051+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:03.934 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:03.932+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:04.006 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:04.004+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:04.974 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:04.973+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:05.006 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:05.004+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:05.989 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:05.988+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:06.009 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:06.005+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:07.006 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:07.005+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:07.021 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:07.020+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:08.024 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:08.022+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:08.049 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:08.048+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:09.065 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:09.063+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:09.099 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:09.097+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:10.027 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:10.026+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:10.052 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:10.051+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:11.053 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:11.051+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:11.099 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:11.097+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:12.065 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:12.063+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:12.126 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:12.124+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:13.086 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:13.084+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:13.119 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:13.116+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:14.057 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:14.055+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:14.132 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:14.130+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:15.055 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:15.053+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:15.120 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:15.118+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:16.018 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:16.016+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:16.162 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:16.159+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:17.024 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:17.021+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:17.190 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:17.187+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:17.988 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:17.986+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:18.207 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:18.205+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:19.024 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:19.022+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:19.190 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:19.188+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:19.997 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:19.995+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:20.199 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:20.197+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:20.970 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:20.964+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:21.152 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:21.149+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:22.016 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:22.014+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:22.152 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:22.149+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:23.025 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:23.023+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:23.181 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:23.178+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:23.980 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:23.978+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:24.209 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:24.207+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:24.939 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:24.937+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:25.161 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:25.158+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:25.915 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:25.912+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:26.159 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:26.157+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:26.945 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:26.943+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:27.190 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:27.188+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:27.976 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:27.974+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:28.227 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:28.225+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:28.986 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:28.983+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:29.189 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:29.186+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:30.001 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:29.999+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:30.206 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:30.204+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:30.981 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:30.979+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:31.171 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:31.169+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:31.984 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:31.982+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:32.159 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:32.157+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:32.988 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:32.985+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:33.121 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:33.119+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:34.030 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:34.028+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:34.412 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:34.077+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:34.989 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:34.987+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:35.083 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:35.080+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:35.983 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:35.981+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:36.096 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:36.094+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:36.949 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:36.947+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:37.048 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:37.046+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:37.960 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:37.958+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:38.032 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:38.029+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:38.924 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:38.922+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:39.016 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:39.014+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:39.890 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:39.888+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:40.015 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:40.012+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:40.940 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:40.938+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:41.050 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:41.048+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:41.898 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:41.896+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:42.059 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:42.057+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:42.941 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:42.939+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:43.094 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:43.092+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:43.906 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:43.905+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:44.048 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:44.046+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:44.858 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:44.857+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:45.069 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:45.067+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:45.845 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:45.843+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:46.109 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:46.107+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:46.860 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:46.858+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:47.117 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:47.115+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:47.865 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:47.863+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:48.140 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:48.139+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:48.877 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:48.876+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:49.105 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:49.104+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:49.917 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:49.916+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:50.131 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:50.130+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:50.873 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:50.871+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:51.145 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:51.144+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:51.833 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:51.831+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:52.131 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:52.129+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:52.811 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:52.810+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:53.180 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:53.179+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:53.855 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:53.854+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:54.222 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:54.220+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:54.849 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:54.847+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:55.257 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:55.256+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:55.837 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:55.836+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:56.304 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:56.302+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:56.867 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:56.866+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:57.347 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:57.346+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:57.863 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:57.862+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:58.328 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:58.327+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:58.865 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:58.863+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:59.300 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:27:59.299+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:27:59.885 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:27:59.883+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:00.261 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:00.260+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:00.859 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:00.856+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:01.250 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:01.248+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:01.855 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:01.853+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:02.228 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:02.226+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:02.823 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:02.821+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:03.245 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:03.242+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:03.847 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:03.844+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:04.209 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:04.207+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:04.799 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:04.797+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:05.255 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:05.253+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:05.843 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:05.841+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:06.206 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:06.204+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:06.858 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:06.856+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:07.200 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:07.198+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:07.886 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:07.884+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:08.239 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:08.237+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:08.869 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:08.866+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:09.266 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:09.264+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:09.827 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:09.825+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:10.260 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:10.257+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:10.830 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:10.827+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:11.215 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:11.213+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:11.806 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:11.803+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:12.209 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:12.207+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:12.830 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:12.827+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:13.192 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:13.189+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:13.847 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:13.845+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:14.308 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:14.231+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:14.808 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:14.806+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:15.261 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:15.259+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:15.811 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:15.809+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:16.223 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:16.221+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:16.853 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:16.850+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:17.196 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:17.194+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:17.828 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:17.826+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:18.211 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:18.208+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:18.825 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:18.823+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:19.227 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:19.224+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:19.806 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:19.804+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:20.189 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:20.186+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:20.787 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:20.785+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:21.196 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:21.194+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:21.740 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:21.738+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:22.230 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:22.228+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:22.740 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:22.738+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:23.275 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:23.273+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:23.733 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:23.730+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:24.302 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:24.299+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:24.753 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:24.750+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:25.268 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:25.265+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:25.724 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:25.722+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:26.289 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:26.286+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:26.729 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:26.727+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:27.331 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:27.329+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:27.688 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:27.686+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:28.293 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:28.291+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:28.715 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:28.712+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:29.303 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:29.301+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:29.743 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:29.740+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:30.285 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:30.282+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:30.705 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:30.703+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:31.326 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:31.323+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:31.701 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:31.699+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:32.345 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:32.343+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:32.717 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:32.715+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:33.350 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:33.349+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:33.703 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:33.702+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:34.302 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:34.300+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:34.720 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:34.718+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:35.292 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:35.291+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:35.690 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:35.689+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:36.268 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:36.267+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:36.738 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:36.737+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:37.252 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:37.250+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:37.782 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:37.781+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:38.216 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:38.215+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:38.767 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:38.766+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:39.211 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:39.209+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:39.720 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:39.718+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:40.198 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:40.197+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:40.724 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:40.723+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:41.204 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:41.203+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:41.732 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:41.730+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:42.225 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:42.224+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:42.746 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:42.744+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:43.204 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:43.203+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:43.744 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:43.743+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:44.205 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:44.203+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:44.753 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:44.751+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:45.227 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:45.226+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:45.718 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:45.717+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:46.268 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:46.267+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:46.741 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:46.740+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:47.286 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:47.285+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:47.704 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:47.703+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:48.284 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:48.283+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:48.668 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:48.667+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:49.236 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:49.235+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:49.693 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:49.691+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:50.191 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:50.189+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:50.667 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:50.666+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:51.173 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:51.171+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:51.686 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:51.683+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:52.191 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:52.189+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:52.641 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:52.638+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:53.216 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:53.214+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:53.637 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:53.635+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:54.185 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:54.182+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:54.679 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:54.677+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:55.149 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:55.147+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:55.700 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:55.698+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:56.102 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:56.100+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:56.700 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:56.697+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:57.146 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:57.144+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:57.693 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:57.690+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:58.112 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:58.110+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:58.680 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:58.678+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:59.114 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:28:59.112+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:28:59.646 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:28:59.644+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:00.121 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:00.118+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:00.642 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:00.640+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:01.097 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:01.094+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:01.595 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:01.593+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:02.078 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:02.076+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:02.614 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:02.612+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:03.054 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:03.051+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:03.633 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:03.631+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:04.010 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:04.007+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:04.677 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:04.674+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:05.015 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:05.013+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:05.703 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:05.700+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:05.988 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:05.986+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:06.725 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:06.723+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:06.950 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:06.948+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:07.735 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:07.733+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:07.977 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:07.975+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:08.739 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:08.736+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:09.022 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:09.019+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:09.780 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:09.778+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:10.068 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:10.066+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:10.787 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:10.785+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:11.095 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:11.093+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:11.837 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:11.835+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:12.069 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:12.066+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:12.800 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:12.798+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:13.058 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:13.055+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:13.849 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:13.846+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:14.032 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:14.030+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:14.893 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:14.891+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:15.000 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:14.998+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:15.854 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:15.852+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:16.004 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:16.001+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:16.852 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:16.850+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:17.047 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:17.045+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:17.830 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:17.828+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:18.061 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:18.059+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:18.843 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:18.840+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:19.025 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:19.022+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:19.878 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:19.876+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:20.009 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:20.007+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:20.926 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:20.924+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:21.037 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:21.034+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:21.952 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:21.950+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:22.054 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:22.051+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:22.950 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:22.948+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:23.038 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:23.035+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:23.976 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:23.975+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:23.988 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:23.987+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:24.944 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:24.943+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:25.002 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:25.000+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:25.980 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:25.978+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:25.990 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:25.989+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:26.951 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:26.950+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:26.976 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:26.974+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:27.971 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:27.969+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:28.014 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:28.012+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:28.971 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:28.970+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:29.022 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:29.020+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:29.999 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:29.998+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:30.002 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:30.000+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:30.976 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:30.974+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:31.033 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:31.031+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:31.936 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:31.935+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:32.019 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:32.018+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:32.941 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:32.939+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:32.983 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:32.981+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:33.896 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:33.895+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:33.987 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:33.986+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:34.910 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:34.909+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:34.975 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:34.974+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:35.880 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:35.879+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:35.996 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:35.995+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:36.892 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:36.891+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:37.012 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:37.010+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:37.876 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:37.875+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:37.966 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:37.964+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:38.879 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:38.878+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:38.980 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:38.979+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:39.858 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:39.857+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:39.989 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:39.987+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:40.852 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:40.850+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:40.994 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:40.991+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:41.868 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:41.866+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:41.952 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:41.949+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:42.890 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:42.887+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:42.919 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:42.917+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:43.921 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:43.919+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:43.962 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:43.960+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:44.920 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:44.918+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:44.994 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:44.991+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:45.872 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:45.870+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:45.952 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:45.950+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:46.870 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:46.868+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:46.945 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:46.943+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:47.886 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:47.883+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:47.978 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:47.975+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:48.892 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:48.890+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:49.014 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:49.012+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:49.867 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:49.865+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:49.992 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:49.990+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:50.819 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:50.817+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:50.991 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:50.988+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:51.812 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:51.810+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:51.958 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:51.956+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:52.855 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:52.853+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:53.002 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:52.999+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:53.905 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:53.903+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:54.025 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:54.023+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:54.926 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:54.919+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:55.042 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:55.040+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:55.906 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:55.904+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:56.026 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:56.023+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:56.909 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:56.907+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:57.027 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:57.025+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:57.890 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:57.887+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:58.016 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:58.014+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:58.902 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:58.899+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:59.024 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:29:59.021+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:29:59.879 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:29:59.877+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:00.031 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:00.028+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:00.886 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:00.883+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:01.050 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:01.047+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:01.919 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:01.917+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:02.085 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:02.083+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:02.886 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:02.884+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:03.112 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:03.109+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:03.927 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:03.924+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:04.094 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:04.091+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:04.966 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:04.964+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:05.078 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:05.076+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:06.008 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:06.006+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:06.080 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:06.077+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:07.013 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:07.011+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:07.040 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:07.037+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:08.042 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:08.040+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:08.043 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:08.040+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:09.000 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:08.998+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:09.090 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:09.088+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:09.981 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:09.978+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:10.140 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:10.138+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:10.991 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:10.988+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:11.094 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:11.092+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:12.039 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:12.036+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:12.057 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:12.055+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:13.010 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:13.008+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:13.048 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:13.046+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:13.965 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:13.963+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:14.018 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:14.016+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:14.918 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:14.917+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:15.047 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:15.045+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:15.907 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:15.906+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:16.016 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:16.014+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:16.876 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:16.875+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:17.046 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:17.045+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:17.833 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:17.831+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:18.065 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:18.064+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:18.862 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:18.861+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:19.027 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:19.025+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:19.884 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:19.882+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:19.995 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:19.994+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:20.846 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:20.845+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:21.033 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:21.031+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:21.833 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:21.831+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:22.027 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:22.025+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:22.798 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:22.797+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:23.050 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:23.049+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:23.822 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:23.820+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:24.011 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:24.009+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:24.799 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:24.798+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:25.058 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:25.057+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:25.790 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:25.789+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:26.022 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:26.020+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:26.777 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:26.776+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:27.033 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:27.031+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:27.808 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:27.807+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:28.072 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:28.071+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:28.859 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:28.858+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:29.065 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:29.064+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:29.829 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:29.828+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:30.066 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:30.065+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:30.859 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:30.858+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:31.097 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:31.096+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:31.906 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:31.905+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:32.070 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:32.067+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:32.912 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:32.910+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:33.103 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:33.100+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:33.900 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:33.897+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:34.069 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:34.066+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:34.864 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:34.862+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:35.090 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:35.087+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:35.854 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:35.852+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:36.103 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:36.100+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:36.821 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:36.819+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:37.143 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:37.141+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:37.866 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:37.864+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:38.105 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:38.102+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:38.824 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:38.822+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:39.087 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:39.085+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:39.864 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:39.861+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:40.116 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:40.114+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:40.904 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:40.902+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:41.068 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:41.066+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:41.906 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:41.903+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:42.042 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:42.040+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:42.897 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:42.895+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:43.069 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:43.066+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:43.887 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:43.885+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:44.116 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:44.113+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:44.871 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:44.869+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:45.165 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:45.163+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:45.895 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:45.893+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:46.135 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:46.132+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:46.863 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:46.860+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:47.090 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:47.087+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:47.861 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:47.859+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:48.133 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:48.129+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:48.836 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:48.834+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:49.085 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:49.083+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:49.852 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:49.850+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:50.041 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:50.039+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:50.809 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:50.807+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:51.001 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:50.999+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:51.853 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:51.851+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:52.017 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:52.015+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:52.825 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:52.822+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:52.995 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:52.993+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:53.846 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:53.844+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:53.998 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:53.996+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:54.879 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:54.877+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:55.035 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:55.032+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:55.845 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:55.843+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:56.010 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:56.007+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:56.847 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:56.845+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:57.021 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:57.018+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:57.854 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:57.852+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:58.071 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:58.068+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:58.879 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:58.877+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:59.043 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:30:59.041+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:30:59.893 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:30:59.890+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:00.038 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:00.035+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:00.869 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:00.867+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:01.066 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:01.063+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:01.839 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:01.837+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:02.073 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:02.070+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:02.889 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:02.887+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:03.031 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:03.029+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:03.926 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:03.925+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:04.013 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:04.010+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:04.976 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:04.974+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:05.023 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:05.021+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:05.976 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:05.975+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:06.045 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:06.043+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:07.014 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:07.012+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:07.016 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:07.015+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:07.999 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:07.998+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:08.018 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:08.017+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:08.995 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:08.993+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:09.023 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:09.021+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:09.968 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:09.967+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:10.004 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:10.003+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:11.011 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:11.010+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:11.025 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:11.024+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:12.015 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:12.013+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:12.028 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:12.027+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:13.013 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:13.012+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:13.033 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:13.032+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:13.989 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:13.988+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:14.033 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:14.031+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:15.006 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:15.005+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:15.060 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:15.059+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:16.022 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:16.020+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:16.099 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:16.097+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:17.018 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:17.017+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:17.121 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:17.120+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:17.999 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:17.997+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:18.119 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:18.118+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:18.975 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:18.973+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:19.169 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:19.168+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:19.991 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:19.990+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:20.202 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:20.201+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:20.953 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:20.952+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:21.219 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:21.218+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:21.951 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:21.949+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:22.227 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:22.226+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:22.921 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:22.918+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:23.196 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:23.194+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:23.910 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:23.908+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:24.233 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:24.230+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:24.935 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:24.933+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:25.191 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:25.188+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:25.898 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:25.896+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:26.229 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:26.226+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:26.900 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:26.897+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:27.187 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:27.185+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:27.906 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:27.903+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:28.196 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:28.193+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:28.899 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:28.896+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:29.245 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:29.242+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:29.901 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:29.899+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:30.293 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:30.291+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:30.952 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:30.949+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:31.256 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:31.254+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:31.990 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:31.987+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:32.213 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:32.211+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:32.978 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:32.976+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:33.198 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:33.196+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:33.951 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:33.949+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:34.165 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:34.162+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:34.973 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:34.970+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:35.191 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:35.189+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:35.998 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:35.996+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:36.229 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:36.227+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:37.029 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:37.026+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:37.239 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:37.237+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:38.005 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:38.003+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:38.241 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:38.239+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:39.012 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:39.010+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:39.230 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:39.228+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:39.983 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:39.980+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:40.186 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:40.181+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:40.945 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:40.942+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:41.155 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:41.153+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:41.909 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:41.906+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:42.153 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:42.151+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:42.907 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:42.904+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:43.193 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:43.191+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:43.920 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:43.917+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:44.209 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:44.207+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:44.893 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:44.891+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:45.215 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:45.213+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:45.864 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:45.861+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:46.167 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:46.165+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:46.862 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:46.860+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:47.125 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:47.123+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:47.908 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:47.906+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:48.162 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:48.160+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:48.888 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:48.886+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:49.146 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:49.143+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:49.846 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:49.844+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:50.152 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:50.150+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:50.805 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:50.802+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:51.147 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:51.145+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:51.766 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:51.764+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:52.117 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:52.114+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:52.771 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:52.769+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:53.073 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:53.070+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:53.816 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:53.814+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:54.105 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:54.103+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:54.829 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:54.827+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:55.059 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:55.058+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:55.831 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:55.830+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:56.064 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:56.062+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:56.828 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:56.826+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:57.061 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:57.060+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:57.846 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:57.845+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:58.023 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:58.022+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:58.852 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:58.851+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:59.063 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:31:59.062+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:31:59.872 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:31:59.871+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:00.077 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:00.075+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:00.857 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:00.855+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:01.029 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:01.028+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:01.871 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:01.869+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:02.072 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:02.070+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:02.831 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:02.829+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:03.056 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:03.054+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:03.873 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:03.872+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:04.045 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:04.044+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:04.892 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:04.890+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:05.033 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:05.032+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:05.932 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:05.931+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:06.076 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:06.075+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:06.945 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:06.944+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:07.083 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:07.082+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:07.954 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:07.952+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:08.042 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:08.040+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:08.941 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:08.939+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:09.014 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:09.012+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:09.917 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:09.916+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:10.063 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:10.062+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:10.959 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:10.958+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:11.039 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:11.038+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:12.002 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:12.001+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:12.032 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:12.031+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:12.983 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:12.981+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:13.049 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:13.047+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:13.955 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:13.953+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:14.091 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:14.089+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:14.994 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:14.992+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:15.061 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:15.058+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:16.037 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:16.035+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:16.075 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:16.072+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:17.030 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:17.028+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:17.041 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:17.038+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:17.984 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:17.982+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:18.029 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:18.027+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:18.939 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:18.937+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:19.035 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:19.032+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:19.944 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:19.942+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:20.016 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:20.013+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:20.900 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:20.898+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:21.033 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:21.031+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:21.882 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:21.879+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:22.055 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:22.052+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:22.906 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:22.904+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:23.092 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:23.090+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:23.949 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:23.947+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:24.105 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:24.102+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:24.925 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:24.923+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:25.091 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:25.088+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:25.895 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:25.892+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:26.127 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:26.125+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:26.919 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:26.916+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:27.121 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:27.118+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:27.967 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:27.965+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:28.171 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:28.169+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:29.005 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:29.003+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:29.142 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:29.140+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:30.021 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:30.018+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:30.183 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:30.181+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:31.022 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:31.020+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:31.231 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:31.229+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:32.031 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:32.028+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:32.193 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:32.191+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:33.009 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:33.006+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:33.232 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:33.230+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:34.057 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:34.055+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:34.260 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:34.257+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:35.076 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:35.073+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:35.274 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:35.271+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:36.041 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:36.039+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:36.259 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:36.257+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:37.067 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:37.064+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:37.309 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:37.307+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:38.019 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:38.017+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:38.331 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:38.329+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:38.981 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:38.979+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:39.284 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:39.282+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:40.004 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:40.001+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:40.322 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:40.320+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:41.046 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:41.043+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:41.335 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:41.333+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:42.072 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:42.070+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:42.382 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:42.380+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:43.105 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:43.103+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:43.358 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:43.356+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:44.128 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:44.126+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:44.392 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:44.389+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:45.170 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:45.168+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:45.408 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:45.406+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:46.142 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:46.140+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:46.415 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:46.413+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:47.106 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:47.104+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:47.379 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:47.378+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:48.064 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:48.063+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:48.386 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:48.384+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:49.049 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:49.048+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:49.372 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:49.371+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:50.070 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:50.068+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:50.357 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:50.355+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:51.095 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:51.093+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:51.400 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:51.399+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:52.098 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:52.097+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:52.354 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:52.352+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:53.124 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:53.123+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:53.320 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:53.318+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:54.144 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:54.142+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:54.286 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:54.284+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:55.110 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:55.109+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:55.328 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:55.326+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:56.118 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:56.117+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:56.299 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:56.298+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:57.114 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:57.113+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:57.306 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:57.305+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:58.069 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:58.068+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:58.314 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:58.313+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:59.047 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:32:59.046+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:32:59.344 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:32:59.343+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:00.060 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:00.059+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:00.300 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:00.299+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:01.105 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:01.103+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:01.286 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:01.284+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:02.122 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:02.121+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:02.274 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:02.273+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:03.104 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:03.103+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:03.289 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:03.288+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:04.130 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:04.127+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:04.278 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:04.276+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:05.170 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:05.167+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:05.276 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:05.274+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:06.182 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:06.179+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:06.256 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:06.254+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:07.148 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:07.146+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:07.266 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:07.264+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:08.111 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:08.109+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:08.308 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:08.305+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:09.070 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:09.068+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:09.344 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:09.342+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:10.073 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:10.070+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:10.359 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:10.357+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:11.090 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:11.088+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:11.389 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:11.387+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:12.108 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:12.106+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:12.354 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:12.352+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:13.089 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:13.086+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:13.342 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:13.340+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:14.059 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:14.057+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:14.329 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:14.327+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:15.095 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:15.092+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:15.298 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:15.296+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:16.118 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:16.116+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:16.253 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:16.250+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:17.129 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:17.127+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:17.303 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:17.300+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:18.114 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:18.112+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:18.334 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:18.332+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:19.140 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:19.138+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:19.369 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:19.366+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:20.185 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:20.183+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:20.388 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:20.385+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:21.207 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:21.205+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:21.414 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:21.412+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:22.181 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:22.178+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:22.405 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:22.403+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:23.186 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:23.184+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:23.473 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:23.422+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:24.156 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:24.154+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:24.444 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:24.441+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:25.159 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:25.157+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:25.412 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:25.410+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:26.140 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:26.138+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:26.367 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:26.364+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:27.117 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:27.114+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:27.366 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:27.363+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:28.082 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:28.080+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:28.335 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:28.332+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:29.095 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:29.093+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:29.295 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:29.292+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:30.095 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:30.093+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:30.344 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:30.342+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:31.146 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:31.143+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:31.380 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:31.378+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:32.156 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:32.154+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:32.378 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:32.376+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:33.185 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:33.182+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:33.374 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:33.372+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:34.138 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:34.135+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:34.371 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:34.368+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:35.131 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:35.128+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:35.371 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:35.369+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:36.165 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:36.163+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:36.370 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:36.368+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:37.188 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:37.187+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:37.354 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:37.353+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:38.183 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:38.181+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:38.357 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:38.356+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:39.185 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:39.184+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:39.310 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:39.309+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:40.138 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:40.137+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:40.296 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:40.295+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:41.142 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:41.141+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:41.259 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:41.257+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:42.118 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:42.116+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:42.267 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:42.266+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:43.084 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:43.083+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:43.242 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:43.241+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:44.082 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:44.080+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:44.241 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:44.240+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:45.059 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:45.058+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:45.228 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:45.227+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:46.063 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:46.062+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:46.212 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:46.211+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:47.082 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:47.081+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:47.224 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:47.223+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:48.105 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:48.103+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:48.218 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:48.216+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:49.149 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:49.148+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:49.184 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:49.182+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:50.124 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:50.123+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:50.147 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:50.146+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:51.149 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:51.148+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:51.163 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:51.162+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:52.116 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:52.114+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:52.177 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:52.175+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:53.100 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:53.099+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:53.158 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:53.157+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:54.117 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:54.115+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:54.124 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:54.122+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:55.092 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:55.090+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:55.115 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:55.113+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:56.066 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:56.064+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:56.129 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:56.127+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:57.068 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:57.066+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:57.167 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:57.165+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:58.114 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:58.112+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:58.196 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:58.194+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:59.066 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:33:59.064+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:33:59.169 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:33:59.167+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:00.056 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:00.054+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:00.134 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:00.132+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:01.026 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:01.024+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:01.130 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:01.127+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:01.997 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:01.995+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:02.133 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:02.130+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:03.009 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:03.006+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:03.108 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:03.106+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:03.979 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:03.977+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:04.145 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:04.143+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:05.019 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:05.017+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:05.130 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:05.127+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:05.988 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:05.986+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:06.138 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:06.136+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:06.943 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:06.941+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:07.137 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:07.135+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:07.909 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:07.907+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:08.132 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:08.129+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:08.891 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:08.888+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:09.107 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:09.105+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:09.909 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:09.907+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:10.078 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:10.076+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:10.864 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:10.861+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:11.124 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:11.122+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:11.815 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:11.813+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:12.100 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:12.098+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:12.766 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:12.764+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:13.063 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:13.061+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:13.748 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:13.745+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:14.092 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:14.089+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:14.758 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:14.756+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:15.130 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:15.128+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:15.763 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:15.761+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:16.110 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:16.108+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:16.771 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:16.769+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:17.080 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:17.077+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:17.738 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:17.736+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:18.092 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:18.089+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:18.748 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:18.746+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:19.127 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:19.125+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:19.772 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:19.769+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:20.134 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:20.132+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:20.770 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:20.768+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:21.183 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:21.181+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:21.798 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:21.795+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:22.214 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:22.212+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:22.771 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:22.769+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:23.233 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:23.230+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:23.812 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:23.809+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:24.267 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:24.265+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:24.772 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:24.770+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:25.276 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:25.274+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:25.739 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:25.737+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:26.259 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:26.257+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:26.766 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:26.764+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:27.276 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:27.274+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:27.814 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:27.813+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:28.255 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:28.254+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:28.828 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:28.827+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:29.272 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:29.270+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:29.802 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:29.801+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:30.276 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:30.275+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:30.789 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:30.787+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:31.260 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:31.258+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:31.819 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:31.818+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:32.307 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:32.305+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:32.772 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:32.770+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:33.319 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:33.317+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:33.763 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:33.762+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:34.354 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:34.353+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:34.731 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:34.730+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:35.354 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:35.352+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:35.765 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:35.764+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:36.394 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:36.392+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:36.762 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:36.760+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:37.364 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:37.363+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:37.752 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:37.751+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:38.410 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:38.409+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:38.742 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:38.740+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:39.361 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:39.360+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:39.772 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:39.771+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:40.386 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:40.385+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:40.778 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:40.777+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:41.350 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:41.349+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:41.829 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:41.827+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:42.303 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:42.301+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:42.878 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:42.877+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:43.266 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:43.265+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:43.919 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:43.917+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:44.275 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:44.273+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:44.900 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:44.898+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:45.299 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:45.296+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:45.869 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:45.866+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:46.333 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:46.331+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:46.914 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:46.912+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:47.286 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:47.284+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:47.930 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:47.927+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:48.292 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:48.290+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:48.895 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:48.893+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:49.343 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:49.340+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:49.918 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:49.916+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:50.313 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:50.311+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:50.932 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:50.930+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:51.336 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:51.334+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:51.901 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:51.899+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:52.323 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:52.320+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:52.918 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:52.916+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:53.305 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:53.302+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:53.903 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:53.901+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:54.295 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:54.293+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:54.897 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:54.895+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:55.309 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:55.307+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:55.906 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:55.904+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:56.320 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:56.318+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:56.931 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:56.929+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:57.292 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:57.289+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:57.944 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:57.942+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:58.254 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:58.252+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:58.977 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:58.974+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:59.253 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:34:59.251+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:34:59.989 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:34:59.987+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:00.295 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:00.293+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:01.034 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:01.032+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:01.287 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:01.285+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:02.054 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:02.051+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:02.247 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:02.245+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:03.093 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:03.090+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:03.203 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:03.201+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:04.092 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:04.090+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:04.166 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:04.164+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:05.123 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:05.121+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:05.124 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:05.122+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:06.080 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:06.078+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:06.156 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:06.153+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:07.053 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:07.051+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:07.152 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:07.150+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:08.082 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:08.080+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:08.124 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:08.121+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:09.092 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:09.089+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:09.120 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:09.118+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:10.084 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:10.081+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:10.146 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:10.144+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:11.070 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:11.067+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:11.181 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:11.178+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:12.034 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:12.031+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:12.221 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:12.218+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:13.055 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:13.052+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:13.198 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:13.196+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:14.091 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:14.090+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:14.195 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:14.193+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:15.125 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:15.123+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:15.175 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:15.173+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:16.140 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:16.138+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:16.164 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:16.163+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:17.167 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:17.165+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:17.194 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:17.193+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:18.145 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:18.143+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:18.164 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:18.163+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:19.166 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:19.164+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:19.169 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:19.168+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:20.151 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:20.150+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:20.193 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:20.192+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:21.166 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:21.165+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:21.169 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:21.168+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:22.149 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:22.147+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:22.192 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:22.191+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:23.118 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:23.116+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:23.237 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:23.235+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:24.088 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:24.087+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:24.241 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:24.239+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:25.122 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:25.121+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:25.230 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:25.229+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:26.116 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:26.115+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:26.267 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:26.266+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:27.110 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:27.109+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:27.255 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:27.254+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:28.061 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:28.060+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:28.282 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:28.281+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:29.054 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:29.053+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:29.327 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:29.326+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:30.071 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:30.069+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:30.350 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:30.348+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:31.083 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:31.082+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:31.359 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:31.358+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:32.071 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:32.070+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:32.372 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:32.370+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:33.111 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:33.110+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:33.338 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:33.337+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:34.114 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:34.113+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:34.363 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:34.362+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:35.081 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:35.079+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:35.345 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:35.343+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:36.082 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:36.080+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:36.376 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:36.373+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:37.124 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:37.121+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:37.366 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:37.364+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:38.142 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:38.140+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:38.401 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:38.399+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:39.161 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:39.158+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:39.396 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:39.394+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:40.196 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:40.193+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:40.360 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:40.357+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:41.240 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:41.238+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:41.340 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:41.338+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:42.250 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:42.248+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:42.332 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:42.330+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:43.228 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:43.225+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:43.348 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:43.346+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:44.243 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:44.241+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:44.345 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:44.343+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:45.274 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:45.272+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:45.314 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:45.311+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:46.260 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:46.258+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:46.282 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:46.280+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:47.248 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:47.246+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:47.278 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:47.276+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:48.202 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:48.199+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:48.246 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:48.244+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:49.198 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:49.195+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:49.219 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:49.217+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:50.214 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:50.212+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:50.251 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:50.249+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:51.238 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:51.236+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:51.292 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:51.289+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:52.203 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:52.201+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:52.298 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:52.296+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:53.177 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:53.174+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:53.334 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:53.332+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:54.220 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:54.218+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:54.299 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:54.297+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:55.226 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:55.224+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:55.308 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:55.305+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:56.233 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:56.231+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:56.308 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:56.306+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:57.262 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:57.260+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:57.275 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:57.273+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:58.255 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:58.252+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:58.290 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:58.288+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:59.225 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:35:59.223+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:35:59.328 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:35:59.326+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:00.206 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:00.203+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:00.353 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:00.351+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:01.194 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:01.192+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:01.317 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:01.315+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:02.153 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:02.151+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:02.368 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:02.366+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:03.112 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:03.110+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:03.402 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:03.399+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:04.079 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:04.078+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:04.383 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:04.381+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:05.061 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:05.059+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:05.414 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:05.412+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:06.108 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:06.106+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:06.431 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:06.429+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:07.067 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:07.065+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:07.413 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:07.410+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:08.074 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:08.073+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:08.429 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:08.428+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:09.058 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:09.057+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:09.400 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:09.398+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:10.012 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:10.011+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:10.448 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:10.447+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:11.023 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:11.022+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:11.470 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:11.469+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:12.033 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:12.032+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:12.442 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:12.441+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:13.014 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:13.012+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:13.472 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:13.471+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:14.052 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:14.051+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:14.449 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:14.448+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:15.011 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:15.009+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:15.472 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:15.471+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:16.004 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:16.003+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:16.426 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:16.425+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:16.956 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:16.955+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:17.474 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:17.473+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:17.938 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:17.936+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:18.516 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:18.514+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:18.922 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:18.920+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:19.469 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:19.468+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:19.898 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:19.897+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:20.434 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:20.433+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:20.910 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:20.908+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:21.417 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:21.416+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:21.871 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:21.870+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:22.466 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:22.465+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:22.888 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:22.887+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:23.505 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:23.503+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:23.874 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:23.873+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:24.505 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:24.504+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:24.920 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:24.918+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:25.525 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:25.523+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:25.957 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:25.955+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:26.482 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:26.479+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:26.970 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:26.968+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:27.495 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:27.492+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:27.956 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:27.954+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:28.535 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:28.533+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:28.928 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:28.926+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:29.551 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:29.549+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:29.974 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:29.972+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:30.520 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:30.518+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:30.991 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:30.988+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:31.550 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:31.548+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:31.971 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:31.969+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:32.537 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:32.535+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:32.985 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:32.983+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:33.577 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:33.575+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:33.965 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:33.963+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:34.554 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:34.552+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:35.000 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:34.998+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:35.581 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:35.578+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:36.007 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:36.005+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:36.572 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:36.570+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:37.039 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:37.036+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:37.528 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:37.526+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:38.087 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:38.085+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:38.548 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:38.546+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:39.042 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:39.040+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:39.548 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:39.546+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:40.038 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:40.035+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:40.549 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:40.547+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:41.070 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:41.068+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:41.541 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:41.539+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:42.105 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:42.103+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:42.589 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:42.586+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:43.127 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:43.125+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:43.562 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:43.560+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:44.119 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:44.117+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:44.518 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:44.516+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:45.169 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:45.167+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:45.503 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:45.500+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:46.197 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:46.195+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:46.489 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:46.486+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:47.213 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:47.211+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:47.518 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:47.516+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:48.244 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:48.242+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:48.563 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:48.561+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:49.224 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:49.222+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:49.584 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:49.582+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:50.264 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:50.262+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:50.567 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:50.565+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:51.289 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:51.286+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:51.557 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:51.555+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:52.264 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:52.262+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:52.597 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:52.595+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:53.269 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:53.267+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:53.619 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:53.617+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:54.222 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:54.220+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:54.604 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:54.602+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:55.271 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:55.269+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:55.560 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:55.558+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:56.311 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:56.309+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:56.523 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:56.521+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:57.272 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:57.270+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:57.517 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:57.515+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:58.278 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:58.277+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:58.469 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:58.467+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:59.310 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:36:59.309+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:36:59.441 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:36:59.440+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:00.357 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:00.356+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:00.427 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:00.426+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:01.393 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:01.392+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:01.404 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:01.403+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:02.395 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:02.394+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:02.447 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:02.446+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:03.401 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:03.399+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:03.427 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:03.426+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:04.389 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:04.387+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:04.473 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:04.472+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:05.398 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:05.397+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:05.438 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:05.437+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:06.361 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:06.360+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:06.452 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:06.451+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:07.411 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:07.409+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:07.444 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:07.443+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:08.368 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:08.367+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:08.417 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:08.416+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:09.396 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:09.395+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:09.430 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:09.429+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:10.423 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:10.422+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:10.468 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:10.467+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:11.395 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:11.394+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:11.494 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:11.493+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:12.348 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:12.347+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:12.457 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:12.456+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:13.341 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:13.339+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:13.460 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:13.459+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:14.337 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:14.334+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:14.425 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:14.424+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:15.305 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:15.303+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:15.418 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:15.417+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:16.297 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:16.295+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:16.415 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:16.412+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:17.303 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:17.301+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:17.462 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:17.459+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:18.319 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:18.317+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:18.419 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:18.417+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:19.274 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:19.272+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:19.379 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:19.377+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:20.233 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:20.231+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:20.390 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:20.388+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:21.259 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:21.257+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:21.361 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:21.359+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:22.294 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:22.292+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:22.402 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:22.400+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:23.294 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:23.291+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:23.410 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:23.408+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:24.283 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:24.281+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:24.441 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:24.438+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:25.236 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:25.234+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:25.471 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:25.469+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:26.278 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:26.276+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:26.508 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:26.506+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:27.245 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:27.243+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:27.540 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:27.538+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:28.224 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:28.221+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:28.495 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:28.493+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:29.198 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:29.196+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:29.529 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:29.527+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:30.177 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:30.175+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:30.484 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:30.482+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:31.175 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:31.173+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:31.471 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:31.469+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:32.161 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:32.158+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:32.423 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:32.421+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:33.208 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:33.206+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:33.407 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:33.405+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:34.235 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:34.233+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:34.391 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:34.389+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:35.217 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:35.215+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:35.407 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:35.405+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:36.226 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:36.224+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:36.419 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:36.417+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:37.252 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:37.250+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:37.463 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:37.461+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:38.297 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:38.295+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:38.494 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:38.492+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:39.293 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:39.291+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:39.533 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:39.531+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:40.288 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:40.286+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:40.565 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:40.563+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:41.277 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:41.274+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:41.570 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:41.567+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:42.273 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:42.271+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:42.538 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:42.536+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:43.313 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:43.310+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:43.571 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:43.569+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:44.321 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:44.318+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:44.620 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:44.618+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:45.280 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:45.278+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:45.651 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:45.649+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:46.289 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:46.286+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:46.658 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:46.656+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:47.300 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:47.297+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:47.612 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:47.610+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:48.306 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:48.304+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:48.649 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:48.647+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:49.295 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:49.294+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:49.613 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:49.611+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:50.321 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:50.320+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:50.579 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:50.578+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:51.336 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:51.335+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:51.604 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:51.602+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:52.321 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:52.319+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:52.639 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:52.638+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:53.316 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:53.315+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:53.689 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:53.688+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:54.346 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:54.345+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:54.657 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:54.654+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:55.352 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:55.351+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:55.634 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:55.632+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:56.379 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:56.378+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:56.683 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:56.681+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:57.333 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:57.332+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:57.673 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:57.672+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:58.295 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:58.294+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:58.653 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:58.652+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:59.309 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:37:59.308+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:37:59.692 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:37:59.691+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:00.307 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:00.306+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:00.689 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:00.688+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:01.290 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:01.289+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:01.656 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:01.655+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:02.338 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:02.337+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:02.616 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:02.615+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:03.389 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:03.387+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:03.646 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:03.645+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:04.359 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:04.358+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:04.620 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:04.619+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:05.341 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:05.340+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:05.644 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:05.642+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:06.309 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:06.308+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:06.681 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:06.679+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:07.289 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:07.287+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:07.701 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:07.699+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:08.311 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:08.309+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:08.671 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:08.669+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:09.288 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:09.286+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:09.698 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:09.696+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:10.292 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:10.290+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:10.674 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:10.672+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:11.264 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:11.262+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:11.707 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:11.704+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:12.305 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:12.303+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:12.709 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:12.707+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:13.340 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:13.337+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:13.689 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:13.687+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:14.358 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:14.356+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:14.676 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:14.674+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:15.406 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:15.404+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:15.640 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:15.638+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:16.390 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:16.388+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:16.610 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:16.608+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:17.373 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:17.371+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:17.572 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:17.570+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:18.389 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:18.387+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:18.599 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:18.596+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:19.344 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:19.342+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:19.621 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:19.619+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:20.378 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:20.376+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:20.588 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:20.586+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:21.417 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:21.415+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:21.545 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:21.543+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:22.391 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:22.389+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:22.543 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:22.541+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:23.433 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:23.431+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:23.534 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:23.532+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:24.392 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:24.389+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:24.509 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:24.507+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:25.425 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:25.423+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:25.551 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:25.549+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:26.407 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:26.405+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:26.591 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:26.589+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:27.414 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:27.412+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:27.571 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:27.569+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:28.420 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:28.418+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:28.589 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:28.586+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:29.416 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:29.414+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:29.630 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:29.628+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:30.376 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:30.374+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:30.634 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:30.632+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:31.404 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:31.402+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:31.647 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:31.645+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:32.416 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:32.414+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:32.694 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:32.692+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:33.369 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:33.366+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:33.698 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:33.696+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:34.381 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:34.379+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:34.691 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:34.689+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:35.400 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:35.398+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:35.643 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:35.641+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:36.369 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:36.367+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:36.692 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:36.690+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:37.351 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:37.349+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:37.696 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:37.693+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:38.362 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:38.359+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:38.693 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:38.691+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:39.333 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:39.331+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:39.670 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:39.669+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:40.336 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:40.335+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:40.642 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:40.641+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:41.339 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:41.338+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:41.641 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:41.640+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:42.296 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:42.295+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:42.603 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:42.602+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:43.318 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:43.317+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:43.653 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:43.652+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:44.302 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:44.301+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:44.683 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:44.682+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:45.346 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:45.345+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:45.668 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:45.667+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:46.302 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:46.301+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:46.640 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:46.639+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:47.338 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:47.337+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:47.626 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:47.625+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:48.376 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:48.374+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:48.661 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:48.660+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:49.350 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:49.349+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:49.686 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:49.685+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:50.311 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:50.310+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:50.640 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:50.639+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:51.264 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:51.263+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:51.684 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:51.683+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:52.295 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:52.294+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:52.685 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:52.684+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:53.254 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:53.252+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:53.659 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:53.658+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:54.227 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:54.226+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:54.669 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:54.668+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:55.224 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:55.223+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:55.691 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:55.690+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:56.225 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:56.224+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:56.661 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:56.660+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:57.208 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:57.207+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:57.657 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:57.655+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:58.198 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:58.196+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:58.703 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:58.701+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:59.211 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:38:59.209+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:38:59.667 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:38:59.665+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:00.221 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:00.219+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:00.706 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:00.704+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:01.206 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:01.204+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:01.721 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:01.719+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:02.240 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:02.238+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:02.756 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:02.754+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:03.204 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:03.202+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:03.720 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:03.718+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:04.238 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:04.236+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:04.717 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:04.715+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:05.207 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:05.205+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:05.705 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:05.703+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:06.226 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:06.224+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:06.734 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:06.732+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:07.275 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:07.273+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:07.762 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:07.760+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:08.316 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:08.314+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:08.796 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:08.794+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:09.272 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:09.270+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:09.759 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:09.757+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:10.247 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:10.245+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:10.806 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:10.804+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:11.202 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:11.200+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:11.842 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:11.840+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:12.182 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:12.180+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:12.847 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:12.845+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:13.201 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:13.199+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:13.816 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:13.814+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:14.239 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:14.237+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:14.830 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:14.828+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:15.264 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:15.262+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:15.822 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:15.820+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:16.252 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:16.250+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:16.832 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:16.830+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:17.246 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:17.244+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:17.851 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:17.849+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:18.260 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:18.258+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:18.858 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:18.856+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:19.285 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:19.283+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:19.844 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:19.841+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:20.295 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:20.293+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:20.831 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:20.829+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:21.293 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:21.291+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:21.861 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:21.859+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:22.262 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:22.260+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:22.895 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:22.893+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:23.310 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:23.308+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:23.886 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:23.884+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:24.277 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:24.275+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:24.847 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:24.845+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:25.298 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:25.296+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:25.886 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:25.884+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:26.342 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:26.340+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:26.859 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:26.857+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:27.321 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:27.319+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:27.888 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:27.886+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:28.367 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:28.365+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:28.874 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:28.872+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:29.382 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:29.380+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:29.921 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:29.919+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:30.341 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:30.340+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:30.904 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:30.903+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:31.363 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:31.362+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:31.932 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:31.931+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:32.371 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:32.369+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:32.891 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:32.889+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:33.385 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:33.384+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:33.856 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:33.854+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:34.356 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:34.355+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:34.808 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:34.807+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:35.388 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:35.387+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:35.820 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:35.819+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:36.405 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:36.404+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:36.781 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:36.780+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:37.362 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:37.361+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:37.758 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:37.757+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:38.340 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:38.339+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:38.719 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:38.718+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:39.337 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:39.328+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:39.737 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:39.736+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:40.294 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:40.293+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:40.703 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:40.702+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:41.259 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:41.258+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:41.660 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:41.659+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:42.212 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:42.211+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:42.662 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:42.661+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:43.225 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:43.224+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:43.687 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:43.686+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:44.217 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:44.216+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:44.678 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:44.677+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:45.239 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:45.238+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:45.649 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:45.648+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:46.195 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:46.194+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:46.682 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:46.681+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:47.171 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:47.170+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:47.640 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:47.639+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:48.125 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:48.122+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:48.609 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:48.606+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:49.098 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:49.096+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:49.608 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:49.606+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:50.091 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:50.089+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:50.572 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:50.570+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:51.103 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:51.101+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:51.541 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:51.539+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:52.068 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:52.066+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:52.568 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:52.566+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:53.026 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:53.023+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:53.578 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:53.576+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:54.021 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:54.019+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:54.595 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:54.593+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:55.002 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:55.000+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:55.562 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:55.560+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:55.994 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:55.992+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:56.562 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:56.560+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:56.975 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:56.973+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:57.608 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:57.606+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:57.992 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:57.989+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:58.559 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:58.557+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:58.976 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:58.974+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:59.534 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:39:59.532+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:39:59.987 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:39:59.985+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:00.540 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:00.538+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:01.026 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:01.024+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:01.574 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:01.572+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:02.074 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:02.072+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:02.609 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:02.607+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:03.036 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:03.034+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:03.577 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:03.575+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:04.002 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:04.000+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:04.587 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:04.585+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:05.023 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:05.021+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:05.556 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:05.553+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:06.002 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:06.000+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:06.524 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:06.521+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:07.004 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:07.002+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:07.505 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:07.503+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:08.010 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:08.008+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:08.503 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:08.501+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:08.990 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:08.988+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:09.493 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:09.491+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:10.011 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:10.009+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:10.538 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:10.535+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:11.004 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:11.002+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:11.565 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:11.563+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:12.015 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:12.013+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:12.534 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:12.532+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:13.022 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:13.020+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:13.572 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:13.570+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:13.997 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:13.995+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:14.561 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:14.559+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:15.005 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:15.003+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:15.607 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:15.604+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:16.012 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:16.010+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:16.589 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:16.586+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:17.048 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:17.046+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:17.580 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:17.578+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:18.071 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:18.069+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:18.569 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:18.567+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:19.116 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:19.114+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:19.608 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:19.606+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:20.148 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:20.146+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:20.590 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:20.588+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:21.123 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:21.122+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:21.572 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:21.571+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:22.091 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:22.090+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:22.559 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:22.558+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:23.077 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:23.075+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:23.592 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:23.591+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:24.112 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:24.111+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:24.577 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:24.576+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:25.092 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:25.091+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:25.587 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:25.586+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:26.116 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:26.115+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:26.548 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:26.547+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:27.161 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:27.160+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:27.547 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:27.546+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:28.186 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:28.185+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:28.587 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:28.586+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:29.157 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:29.156+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:29.542 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:29.541+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:30.142 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:30.140+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:30.573 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:30.572+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:31.185 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:31.184+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:31.621 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:31.620+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:32.195 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:32.194+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:32.636 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:32.635+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:33.211 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:33.210+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:33.600 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:33.599+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:34.203 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:34.202+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:34.629 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:34.628+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:35.197 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:35.196+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:35.582 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:35.581+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:36.155 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:36.154+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:36.568 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:36.567+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:37.189 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:37.188+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:37.617 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:37.616+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:38.171 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:38.170+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:38.610 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:38.609+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:39.164 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:39.162+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:39.658 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:39.656+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:40.188 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:40.186+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:40.684 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:40.682+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:41.153 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:41.151+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:41.645 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:41.643+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:42.185 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:42.183+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:42.676 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:42.674+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:43.219 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:43.217+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:43.672 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:43.670+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:44.247 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:44.245+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:44.676 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:44.674+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:45.215 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:45.213+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:45.717 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:45.715+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:46.208 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:46.206+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:46.711 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:46.709+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:47.185 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:47.183+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:47.742 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:47.740+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:48.154 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:48.152+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:48.793 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:48.791+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:49.134 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:49.132+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:49.763 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:49.760+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:50.155 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:50.153+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:50.794 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:50.792+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:51.163 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:51.161+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:51.836 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:51.834+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:52.150 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:52.148+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:52.869 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:52.867+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:53.112 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:53.110+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:53.864 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:53.862+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:54.094 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:54.093+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:54.842 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:54.840+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:55.063 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:55.060+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:55.872 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:55.871+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:56.094 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:56.092+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:56.835 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:56.833+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:57.057 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:57.055+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:57.860 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:57.859+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:58.068 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:58.067+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:58.847 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:58.845+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:59.035 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:59.033+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:59.882 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:40:59.880+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:40:59.990 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:40:59.988+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:00.896 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:00.894+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:00.950 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:00.948+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:01.885 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:01.883+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:02.001 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:01.999+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:02.870 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:02.868+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:02.989 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:02.987+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:03.918 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:03.917+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:03.979 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:03.977+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:04.948 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:04.946+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:04.969 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:04.967+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:05.930 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:05.928+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:05.934 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:05.932+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:06.914 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:06.912+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:06.975 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:06.973+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:07.915 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:07.913+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:07.958 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:07.957+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:08.938 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:08.936+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:08.944 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:08.942+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:09.935 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:09.933+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:09.954 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:09.953+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:10.945 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:10.943+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:11.004 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:11.002+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:11.926 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:11.925+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:11.986 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:11.985+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:12.942 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:12.941+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:13.028 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:13.027+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:13.956 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:13.955+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:14.070 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:14.069+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:14.924 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:14.923+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:15.040 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:15.036+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:15.887 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:15.886+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:16.073 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:16.072+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:16.904 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:16.902+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:17.075 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:17.074+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:17.949 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:17.948+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:18.119 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:18.117+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:18.949 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:18.948+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:19.082 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:19.081+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:19.909 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:19.907+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:20.122 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:20.121+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:20.916 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:20.915+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:21.153 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:21.152+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:21.867 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:21.866+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:22.186 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:22.185+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:22.905 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:22.904+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:23.164 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:23.163+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:23.897 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:23.896+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:24.116 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:24.115+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:24.872 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:24.871+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:25.110 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:25.109+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:25.859 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:25.858+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:26.132 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:26.131+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:26.853 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:26.852+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:27.104 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:27.103+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:27.880 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:27.879+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:28.145 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:28.144+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:28.908 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:28.907+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:29.144 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:29.143+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:29.860 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:29.858+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:30.148 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:30.146+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:30.812 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:30.810+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:31.119 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:31.117+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:31.812 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:31.810+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:32.157 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:32.155+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:32.793 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:32.791+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:33.116 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:33.114+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:33.790 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:33.788+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:34.096 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:34.094+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:34.752 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:34.750+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:35.145 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:35.143+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:35.762 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:35.760+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:36.166 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:36.164+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:36.773 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:36.770+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:37.152 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:37.150+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:37.816 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:37.814+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:38.134 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:38.132+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:38.844 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:38.842+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:39.132 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:39.130+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:39.878 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:39.876+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:40.104 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:40.102+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:40.848 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:40.846+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:41.134 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:41.132+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:41.895 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:41.893+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:42.091 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:42.089+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:42.931 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:42.929+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:43.060 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:43.058+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:43.898 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:43.896+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:44.066 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:44.064+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:44.872 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:44.871+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:45.029 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:45.027+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:45.880 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:45.878+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:45.990 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:45.988+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:46.868 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:46.866+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:46.952 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:46.950+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:47.842 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:47.840+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:47.928 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:47.926+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:48.824 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:48.821+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:48.952 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:48.950+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:49.834 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:49.832+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:49.951 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:49.949+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:50.810 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:50.808+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:50.922 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:50.920+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:51.818 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:51.816+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:51.881 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:51.880+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:52.859 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:52.858+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:52.904 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:52.902+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:53.869 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:53.867+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:53.923 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:53.921+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:54.888 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:54.886+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:54.909 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:54.907+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:55.862 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:55.860+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:55.936 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:55.934+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:56.904 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:56.902+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:56.909 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:56.907+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:57.915 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:57.914+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:57.928 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:57.927+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:58.880 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:58.878+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:58.956 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:58.954+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:59.921 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:41:59.918+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:41:59.935 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:41:59.933+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:00.898 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:00.896+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:00.943 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:00.941+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:01.851 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:01.849+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:01.895 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:01.893+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:02.885 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:02.884+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:02.894 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:02.893+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:03.861 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:03.860+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:03.918 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:03.917+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:04.873 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:04.872+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:04.931 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:04.930+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:05.883 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:05.882+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:05.969 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:05.968+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:06.917 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:06.916+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:07.017 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:07.016+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:07.949 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:07.948+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:08.060 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:08.059+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:08.963 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:08.963+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:09.073 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:09.072+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:09.943 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:09.942+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:10.093 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:10.092+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:10.933 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:10.932+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:11.097 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:11.096+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:11.958 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:11.957+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:12.103 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:12.102+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:12.990 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:12.989+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:13.151 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:13.150+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:14.012 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:14.011+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:14.119 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:14.118+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:15.058 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:15.057+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:15.112 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:15.111+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:16.073 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:16.071+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:16.103 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:16.102+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:17.070 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:17.069+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:17.111 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:17.110+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:18.047 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:18.045+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:18.067 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:18.066+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:19.025 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:19.024+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:19.029 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:19.029+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:19.984 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:19.982+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:20.032 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:20.030+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:20.975 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:20.972+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:21.006 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:21.004+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:21.959 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:21.957+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:21.977 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:21.975+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:22.919 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:22.917+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:23.006 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:23.004+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:23.922 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:23.920+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:24.035 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:24.033+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:24.947 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:24.945+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:25.063 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:25.061+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:25.935 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:25.933+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:26.084 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:26.083+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:26.957 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:26.955+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:27.041 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:27.039+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:27.965 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:27.963+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:28.061 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:28.059+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:28.930 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:28.928+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:29.022 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:29.020+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:29.938 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:29.936+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:29.990 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:29.988+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:30.901 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:30.899+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:31.010 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:31.008+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:31.900 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:31.898+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:32.002 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:32.000+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:32.894 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:32.892+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:33.053 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:33.051+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:33.891 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:33.889+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:34.078 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:34.076+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:34.885 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:34.883+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:35.073 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:35.071+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:35.877 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:35.875+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:36.101 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:36.099+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:36.885 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:36.883+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:37.114 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:37.113+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:37.882 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:37.880+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:38.126 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:38.125+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:38.850 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:38.849+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:39.134 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:39.132+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:39.825 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:39.824+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:40.121 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:40.119+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:40.778 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:40.777+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:41.116 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:41.114+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:41.817 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:41.815+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:42.086 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:42.085+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:42.797 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:42.796+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:43.127 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:43.125+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:43.775 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:43.773+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:44.131 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:44.129+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:44.796 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:44.794+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:45.121 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:45.119+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:45.823 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:45.821+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:46.128 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:46.126+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:46.805 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:46.803+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:47.173 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:47.171+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:47.811 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:47.809+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:48.155 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:48.153+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:48.813 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:48.811+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:49.149 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:49.147+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:49.805 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:49.803+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:50.186 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:50.184+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:50.810 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:50.808+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:51.222 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:51.220+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:51.768 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:51.766+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:52.233 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:52.232+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:52.735 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:52.734+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:53.262 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:53.261+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:53.755 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:53.754+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:54.271 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:54.269+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:54.761 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:54.760+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:55.253 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:55.252+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:55.715 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:55.714+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:56.264 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:56.263+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:56.700 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:56.699+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:57.253 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:57.252+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:57.712 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:57.711+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:58.242 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:58.241+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:58.744 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:58.743+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:59.243 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:42:59.242+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:42:59.721 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:42:59.719+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:00.264 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:00.263+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:00.764 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:00.763+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:01.232 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:01.231+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:01.744 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:01.743+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:02.277 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:02.276+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:02.780 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:02.779+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:03.237 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:03.236+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:03.751 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:03.750+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:04.280 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:04.279+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:04.745 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:04.744+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:05.320 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:05.319+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:05.758 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:05.757+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:06.330 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:06.329+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:06.759 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:06.758+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:07.351 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:07.350+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:07.734 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:07.733+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:08.354 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:08.353+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:08.685 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:08.684+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:09.373 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:09.372+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:09.655 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:09.654+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:10.346 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:10.344+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:10.697 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:10.696+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:11.367 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:11.365+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:11.723 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:11.721+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:12.346 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:12.344+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:12.727 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:12.725+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:13.344 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:13.342+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:13.774 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:13.773+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:14.383 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:14.381+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:14.757 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:14.755+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:15.373 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:15.371+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:15.792 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:15.790+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:16.344 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:16.342+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:16.838 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:16.836+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:17.382 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:17.380+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:17.850 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:17.848+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:18.402 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:18.400+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:18.853 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:18.851+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:19.394 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:19.393+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:19.822 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:19.820+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:20.348 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:20.346+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:20.781 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:20.779+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:21.381 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:21.379+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:21.819 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:21.817+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:22.403 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:22.401+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:22.791 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:22.789+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:23.356 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:23.355+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:23.758 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:23.756+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:24.386 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:24.384+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:24.719 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:24.718+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:25.427 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:25.425+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:25.699 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:25.697+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:26.408 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:26.406+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:26.670 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:26.668+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:27.367 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:27.365+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:27.674 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:27.672+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:28.325 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:28.323+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:28.685 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:28.683+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:29.314 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:29.312+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:29.667 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:29.665+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:30.358 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:30.356+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:30.630 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:30.628+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:31.374 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:31.372+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:31.606 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:31.604+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:32.342 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:32.340+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:32.575 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:32.573+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:33.306 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:33.304+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:33.537 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:33.535+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:34.275 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:34.273+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:34.557 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:34.556+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:35.232 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:35.230+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:35.606 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:35.605+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:36.185 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:36.183+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:36.622 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:36.620+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:37.170 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:37.168+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:37.614 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:37.612+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:38.203 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:38.201+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:38.605 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:38.603+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:39.248 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:39.247+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:39.565 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:39.564+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:40.221 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:40.219+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:40.520 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:40.518+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:41.244 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:41.242+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:41.480 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:41.478+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:42.215 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:42.213+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:42.461 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:42.459+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:43.209 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:43.208+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:43.463 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:43.463+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:44.221 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:44.220+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:44.430 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:44.429+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:45.204 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:45.203+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:45.455 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:45.454+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:46.197 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:46.196+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:46.465 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:46.464+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:47.213 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:47.212+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:47.468 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:47.467+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:48.222 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:48.221+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:48.419 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:48.418+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:49.241 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:49.241+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:49.431 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:49.430+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:50.282 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:50.281+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:50.421 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:50.420+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:51.243 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:51.242+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:51.403 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:51.402+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:52.265 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:52.264+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:52.412 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:52.411+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:53.218 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:53.217+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:53.387 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:53.386+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:54.214 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:54.213+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:54.394 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:54.393+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:55.165 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:55.164+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:55.368 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:55.367+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:56.145 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:56.144+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:56.353 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:56.352+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:57.182 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:57.181+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:57.348 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:57.347+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:58.177 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:58.176+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:58.310 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:58.309+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:59.205 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:43:59.204+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:43:59.353 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:43:59.353+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:00.236 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:00.235+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:00.396 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:00.395+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:01.187 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:01.185+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:01.412 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:01.410+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:02.198 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:02.196+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:02.439 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:02.437+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:03.239 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:03.238+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:03.489 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:03.487+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:04.252 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:04.250+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:04.500 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:04.498+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:05.214 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:05.212+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:05.534 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:05.532+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:06.240 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:06.238+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:06.566 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:06.564+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:07.250 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:07.248+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:07.606 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:07.602+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:08.222 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:08.220+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:08.566 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:08.564+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:09.260 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:09.258+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:09.582 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:09.580+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:10.223 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:10.221+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:10.624 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:10.622+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:11.272 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:11.270+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:11.602 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:11.600+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:12.308 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:12.306+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:12.605 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:12.603+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:13.322 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:13.320+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:13.633 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:13.631+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:14.365 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:14.363+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:14.648 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:14.646+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:15.400 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:15.398+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:15.653 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:15.651+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:16.362 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:16.360+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:16.629 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:16.627+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:17.406 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:17.404+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:17.668 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:17.666+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:18.407 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:18.405+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:18.690 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:18.688+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:19.416 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:19.414+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:19.643 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:19.641+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:20.436 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:20.434+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:20.666 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:20.664+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:21.440 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:21.438+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:21.696 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:21.694+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:22.411 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:22.409+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:22.746 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:22.745+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:23.404 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:23.402+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:23.791 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:23.789+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:24.372 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:24.370+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:24.761 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:24.759+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:25.382 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:25.380+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:25.742 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:25.740+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:26.341 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:26.340+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:26.708 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:26.706+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:27.330 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:27.328+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:27.719 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:27.717+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:28.322 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:28.320+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:28.745 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:28.743+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:29.351 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:29.349+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:29.749 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:29.747+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:30.346 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:30.344+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:30.794 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:30.793+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:31.320 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:31.318+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:31.830 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:31.828+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:32.295 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:32.293+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:32.849 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:32.848+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:33.328 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:33.326+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:33.843 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:33.842+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:34.335 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:34.334+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:34.817 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:34.816+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:35.351 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:35.350+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:35.793 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:35.792+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:36.315 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:36.314+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:36.808 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:36.807+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:37.293 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:37.292+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:37.832 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:37.831+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:38.296 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:38.296+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:38.787 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:38.786+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:39.301 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:39.300+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:39.753 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:39.752+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:40.292 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:40.291+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:40.750 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:40.749+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:41.261 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:41.260+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:41.738 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:41.737+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:42.282 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:42.281+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:42.707 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:42.706+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:43.289 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:43.288+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:43.710 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:43.709+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:44.303 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:44.302+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:44.703 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:44.702+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:45.281 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:45.280+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:45.662 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:45.661+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:46.249 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:46.248+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:46.710 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:46.709+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:47.225 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:47.224+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:47.713 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:47.712+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:48.247 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:48.246+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:48.720 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:48.720+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:49.276 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:49.275+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:49.698 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:49.698+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:50.303 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:50.302+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:50.726 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:50.726+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:51.299 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:51.297+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:51.741 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:51.740+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:52.308 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:52.306+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:52.738 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:52.736+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:53.280 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:53.278+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:53.713 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:53.711+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:54.271 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:54.269+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:54.676 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:54.675+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:55.294 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:55.293+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:55.673 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:55.671+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:56.258 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:56.256+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:56.679 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:56.677+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:57.239 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:57.237+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:57.724 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:57.722+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:58.231 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:58.229+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:58.696 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:58.694+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:59.217 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:44:59.215+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:44:59.735 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:44:59.733+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:00.237 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:00.235+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:00.769 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:00.768+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:01.252 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:01.250+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:01.754 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:01.752+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:02.289 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:02.287+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:02.744 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:02.742+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:03.323 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:03.321+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:03.709 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:03.707+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:04.366 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:04.364+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:04.665 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:04.663+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:05.321 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:05.319+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:05.712 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:05.710+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:06.272 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:06.271+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:06.668 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:06.666+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:07.293 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:07.292+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:07.712 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:07.710+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:08.310 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:08.308+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:08.731 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:08.729+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:09.336 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:09.334+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:09.724 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:09.723+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:10.379 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:10.378+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:10.739 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:10.737+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:11.330 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:11.328+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:11.779 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:11.777+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:12.286 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:12.284+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:12.792 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:12.790+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:13.328 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:13.326+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:13.813 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:13.812+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:14.350 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:14.348+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:14.833 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:14.831+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:15.318 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:15.316+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:15.837 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:15.835+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:16.367 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:16.366+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:16.833 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:16.831+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:17.348 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:17.346+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:17.818 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:17.816+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:18.380 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:18.378+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:18.853 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:18.851+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:19.376 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:19.374+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:19.862 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:19.860+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:20.365 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:20.363+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:20.896 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:20.894+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:21.370 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:21.368+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:21.902 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:21.900+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:22.351 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:22.349+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:22.943 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:22.941+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:23.349 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:23.347+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:23.908 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:23.907+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:24.324 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:24.323+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:24.943 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:24.942+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:25.282 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:25.282+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:25.966 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:25.965+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:26.296 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:26.295+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:26.988 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:26.987+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:27.321 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:27.320+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:27.940 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:27.940+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:28.291 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:28.290+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:28.982 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:28.982+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:29.327 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:29.326+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:29.988 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:29.987+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:30.331 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:30.330+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:30.990 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:30.989+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:31.361 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:31.360+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:31.950 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:31.949+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:32.382 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:32.381+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:32.991 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:32.991+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:33.365 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:33.364+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:34.015 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:34.015+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:34.335 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:34.334+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:35.015 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:35.014+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:35.357 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:35.356+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:36.017 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:36.016+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:36.339 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:36.338+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:37.030 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:37.030+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:37.307 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:37.305+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:38.053 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:38.052+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:38.351 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:38.351+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:39.046 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:39.045+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:39.400 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:39.399+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:40.065 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:40.063+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:40.419 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:40.418+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:41.020 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:41.019+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:41.391 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:41.390+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:42.024 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:42.022+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:42.365 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:42.363+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:43.062 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:43.060+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:43.318 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:43.317+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:44.054 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:44.052+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:44.362 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:44.360+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:45.015 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:45.013+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:45.363 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:45.361+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:45.977 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:45.975+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:46.340 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:46.339+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:46.929 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:46.927+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:47.369 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:47.367+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:47.959 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:47.958+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:48.361 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:48.359+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:48.951 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:48.950+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:49.324 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:49.322+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:49.943 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:49.941+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:50.290 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:50.288+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:50.914 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:50.912+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:51.266 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:51.264+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:51.910 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:51.908+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:52.227 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:52.226+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:52.903 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:52.901+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:53.215 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:53.213+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:53.906 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:53.905+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:54.206 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:54.204+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:54.923 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:54.921+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:55.184 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:55.182+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:55.965 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:55.963+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:56.174 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:56.173+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:56.922 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:56.920+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:57.160 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:57.159+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:57.895 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:57.894+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:58.162 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:58.160+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:58.905 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:58.903+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:59.151 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:45:59.149+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:45:59.918 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:45:59.916+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:00.103 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:00.101+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:00.969 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:00.966+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:01.127 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:01.125+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:02.018 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:02.016+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:02.092 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:02.090+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:03.047 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:03.045+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:03.076 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:03.075+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:04.032 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:04.030+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:04.113 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:04.111+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:05.053 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:05.051+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:05.076 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:05.074+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:06.046 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:06.044+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:06.074 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:06.072+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:07.058 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:07.056+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:07.071 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:07.069+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:08.047 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:08.045+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:08.071 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:08.069+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:09.052 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:09.050+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:09.106 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:09.105+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:10.017 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:10.016+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:10.144 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:10.142+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:11.027 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:11.025+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:11.169 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:11.167+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:12.076 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:12.074+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:12.129 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:12.127+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:13.032 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:13.030+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:13.160 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:13.158+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:14.029 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:14.027+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:14.122 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:14.122+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:15.003 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:15.002+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:15.148 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:15.147+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:15.978 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:15.977+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:16.181 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:16.180+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:16.978 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:16.977+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:17.168 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:17.167+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:18.018 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:18.018+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:18.163 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:18.162+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:19.046 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:19.045+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:19.183 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:19.182+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:20.042 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:20.041+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:20.179 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:20.178+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:21.040 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:21.039+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:21.150 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:21.150+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:22.018 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:22.017+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:22.186 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:22.185+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:22.971 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:22.970+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:23.138 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:23.137+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:23.980 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:23.979+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:24.188 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:24.187+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:24.957 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:24.956+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:25.222 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:25.222+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:25.932 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:25.932+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:26.228 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:26.227+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:26.923 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:26.922+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:27.201 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:27.200+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:27.926 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:27.926+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:28.193 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:28.192+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:28.960 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:28.959+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:29.218 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:29.217+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:29.956 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:29.955+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:30.255 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:30.254+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:30.983 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:30.982+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:31.206 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:31.206+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:32.032 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:32.031+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:32.215 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:32.215+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:33.050 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:33.048+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:33.251 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:33.249+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:34.095 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:34.093+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:34.203 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:34.201+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:35.055 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:35.054+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:35.224 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:35.223+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:36.045 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:36.043+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:36.251 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:36.250+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:37.005 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:37.003+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:37.239 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:37.237+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:38.018 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:38.016+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:38.227 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:38.225+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:39.010 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:39.008+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:39.257 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:39.256+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:39.968 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:39.966+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:40.267 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:40.265+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:40.936 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:40.934+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:41.307 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:41.306+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:41.957 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:41.955+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:42.294 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:42.292+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:43.002 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:43.000+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:43.303 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:43.301+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:43.982 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:43.980+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:44.290 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:44.289+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:45.025 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:45.023+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:45.297 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:45.295+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:46.008 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:46.006+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:46.337 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:46.336+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:46.971 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:46.969+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:47.341 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:47.339+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:47.969 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:47.967+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:48.365 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:48.363+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:49.015 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:49.013+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:49.321 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:49.319+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:50.007 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:50.005+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:50.321 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:50.319+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:50.963 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:50.962+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:51.323 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:51.321+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:51.979 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:51.977+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:52.317 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:52.315+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:52.960 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:52.958+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:53.329 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:53.327+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:53.972 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:53.971+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:54.317 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:54.315+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:54.991 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:54.989+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:55.320 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:55.318+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:56.021 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:56.019+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:56.333 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:56.331+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:56.996 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:56.995+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:57.286 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:57.284+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:57.958 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:57.957+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:58.333 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:58.331+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:58.926 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:58.924+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:59.379 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:46:59.377+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:46:59.946 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:46:59.945+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:00.341 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:00.339+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:00.974 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:00.972+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:01.334 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:01.332+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:01.947 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:01.945+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:02.303 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:02.301+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:02.930 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:02.928+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:03.256 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:03.254+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:03.901 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:03.899+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:04.228 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:04.226+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:04.900 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:04.898+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:05.228 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:05.227+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:05.858 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:05.857+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:06.271 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:06.270+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:06.855 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:06.855+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:07.307 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:07.306+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:07.903 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:07.902+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:08.268 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:08.268+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:08.926 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:08.925+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:09.319 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:09.318+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:09.914 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:09.913+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:10.309 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:10.309+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:10.884 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:10.883+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:11.342 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:11.341+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:11.908 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:11.907+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:12.357 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:12.357+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:12.955 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:12.955+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:13.316 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:13.315+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:13.993 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:13.992+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:14.269 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:14.268+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:15.029 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:15.028+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:15.228 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:15.227+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:15.986 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:15.985+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:16.208 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:16.207+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:16.973 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:16.972+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:17.219 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:17.219+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:17.993 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:17.992+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:18.232 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:18.231+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:18.986 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:18.985+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:19.267 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:19.266+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:19.985 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:19.984+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:20.249 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:20.248+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:21.025 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:21.024+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:21.286 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:21.286+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:22.064 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:22.063+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:22.335 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:22.335+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:23.102 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:23.100+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:23.313 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:23.311+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:24.079 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:24.077+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:24.334 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:24.332+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:25.087 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:25.085+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:25.342 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:25.340+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:26.088 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:26.086+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:26.301 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:26.299+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:27.058 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:27.056+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:27.253 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:27.251+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:28.095 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:28.093+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:28.208 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:28.206+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:29.056 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:29.054+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:29.225 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:29.223+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:30.066 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:30.064+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:30.196 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:30.194+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:31.023 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:31.021+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:31.184 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:31.183+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:32.003 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:32.001+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:32.197 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:32.195+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:32.991 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:32.990+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:33.245 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:33.243+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:33.976 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:33.974+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:34.268 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:34.267+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:34.950 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:34.948+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:35.259 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:35.258+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:35.977 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:35.975+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:36.217 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:36.216+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:36.964 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:36.962+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:37.188 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:37.186+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:37.979 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:37.977+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:38.182 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:38.180+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:39.013 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:39.011+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:39.140 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:39.138+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:40.033 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:40.031+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:40.133 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:40.131+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:40.997 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:40.995+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:41.093 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:41.091+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:41.952 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:41.950+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:42.119 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:42.118+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:42.992 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:42.990+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:43.108 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:43.106+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:44.008 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:44.007+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:44.153 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:44.150+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:45.006 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:45.004+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:45.113 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:45.111+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:46.040 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:46.038+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:46.147 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:46.146+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:47.016 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:47.014+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:47.106 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:47.104+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:48.062 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:48.060+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:48.065 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:48.063+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:49.067 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:49.065+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:49.097 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:49.095+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:50.045 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:50.044+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:50.081 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:50.080+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:51.093 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:51.091+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:51.116 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:51.114+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:52.123 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:52.122+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:52.134 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:52.132+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:53.150 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:53.148+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:53.169 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:53.167+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:54.144 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:54.142+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:54.183 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:54.181+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:55.100 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:55.098+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:55.218 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:55.216+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:56.094 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:56.093+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:56.184 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:56.184+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:57.135 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:57.135+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:57.196 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:57.196+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:58.148 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:58.147+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:58.156 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:58.156+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:59.165 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:47:59.164+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:47:59.180 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:47:59.180+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:00.198 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:00.198+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:00.225 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:00.224+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:01.236 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:01.235+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:01.253 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:01.253+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:02.263 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:02.262+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:02.295 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:02.294+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:03.259 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:03.258+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:03.260 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:03.260+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:04.229 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:04.228+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:04.230 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:04.230+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:05.221 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:05.220+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:05.237 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:05.236+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:06.179 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:06.178+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:06.230 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:06.229+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:07.229 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:07.228+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:07.237 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:07.236+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:08.213 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:08.212+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:08.287 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:08.286+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:09.193 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:09.192+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:09.289 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:09.288+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:10.158 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:10.157+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:10.246 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:10.246+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:11.114 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:11.114+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:11.240 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:11.239+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:12.141 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:12.140+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:12.214 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:12.214+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:13.125 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:13.122+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:13.219 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:13.218+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:14.150 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:14.149+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:14.184 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:14.182+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:15.108 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:15.106+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:15.193 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:15.191+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:16.146 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:16.144+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:16.235 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:16.233+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:17.103 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:17.102+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:17.199 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:17.197+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:18.146 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:18.145+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:18.155 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:18.153+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:19.141 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:19.139+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:19.181 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:19.179+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:20.162 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:20.160+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:20.228 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:20.226+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:21.174 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:21.172+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:21.270 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:21.269+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:22.141 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:22.139+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:22.260 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:22.258+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:23.184 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:23.182+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:23.296 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:23.294+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:24.161 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:24.159+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:24.313 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:24.311+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:25.141 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:25.139+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:25.297 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:25.296+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:26.152 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:26.150+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:26.254 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:26.251+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:27.135 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:27.134+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:27.263 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:27.261+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:28.186 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:28.184+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:28.266 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:28.265+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:29.194 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:29.193+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:29.242 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:29.240+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:30.227 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:30.225+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:30.277 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:30.275+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:31.205 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:31.204+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:31.282 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:31.280+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:32.239 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:32.237+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:32.241 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:32.239+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:33.210 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:33.209+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:33.222 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:33.221+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:34.250 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:34.248+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:34.259 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:34.258+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:35.220 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:35.218+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:35.232 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:35.231+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:36.183 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:36.181+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:36.206 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:36.204+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:37.175 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:37.174+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:37.194 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:37.193+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:38.173 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:38.171+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:38.187 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:38.185+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:39.142 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:39.140+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:39.179 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:39.177+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:40.131 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:40.129+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:40.142 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:40.141+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:41.108 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:41.107+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:41.135 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:41.133+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:42.150 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:42.148+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:42.171 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:42.169+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:43.101 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:43.099+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:43.129 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:43.127+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:44.143 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:44.141+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:44.146 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:44.145+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:45.142 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:45.140+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:45.150 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:45.149+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:46.149 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:46.147+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:46.173 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:46.171+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:47.117 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:47.117+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:47.142 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:47.141+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:48.086 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:48.085+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:48.134 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:48.133+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:49.051 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:49.050+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:49.163 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:49.163+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:50.089 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:50.089+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:50.139 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:50.139+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:51.121 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:51.121+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:51.138 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:51.138+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:52.109 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:52.107+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:52.161 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:52.160+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:53.125 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:53.124+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:53.145 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:53.144+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:54.095 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:54.094+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:54.140 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:54.139+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:55.069 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:55.068+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:55.121 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:55.120+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:56.083 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:56.082+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:56.165 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:56.165+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:57.111 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:57.111+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:57.119 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:57.119+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:58.107 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:58.106+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:58.153 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:58.152+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:59.071 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:48:59.071+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:48:59.157 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:48:59.156+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:00.061 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:00.060+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:00.196 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:00.195+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:01.085 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:01.084+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:01.194 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:01.193+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:02.110 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:02.110+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:02.165 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:02.164+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:03.140 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:03.139+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:03.195 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:03.194+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:04.103 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:04.102+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:04.224 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:04.222+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:05.058 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:05.056+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:05.237 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:05.235+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:06.036 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:06.034+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:06.245 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:06.243+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:07.028 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:07.026+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:07.270 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:07.268+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:08.008 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:08.006+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:08.298 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:08.296+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:09.050 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:09.048+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:09.307 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:09.305+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:09.867 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T17:49:09.865+0000 7fd64b457700 0 -- 172.21.15.82:0/3601686610 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7fd6300366d0 msgr2=0x7fd63003ef50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-18T17:49:09.868 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T17:49:09.866+0000 7fd64b457700 0 -- 172.21.15.82:0/3601686610 >> [v2:172.21.15.82:6816/87291,v1:172.21.15.82:6817/87291] conn(0x7fd630030040 msgr2=0x7fd63004fc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-18T17:49:10.047 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:10.045+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:10.336 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:10.334+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:11.052 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:11.050+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:11.297 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:11.295+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:12.083 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:12.081+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:12.254 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:12.252+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:13.107 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:13.106+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:13.234 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:13.233+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:14.081 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:14.079+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:14.202 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:14.201+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:15.092 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:15.090+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:15.226 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:15.224+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:16.092 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:16.091+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:16.201 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:16.200+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:17.130 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:17.128+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:17.199 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:17.197+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:18.094 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:18.092+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:18.186 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:18.184+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:19.079 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:19.077+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:19.140 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:19.138+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:20.101 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:20.100+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:20.102 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:20.100+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:21.094 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:21.092+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:21.144 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:21.142+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:22.067 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:22.065+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:22.172 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:22.171+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:23.083 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:23.081+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:23.190 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:23.188+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:24.078 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:24.076+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:24.170 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:24.169+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:25.125 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:25.124+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:25.200 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:25.198+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:26.144 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:26.142+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:26.208 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:26.206+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:27.119 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:27.118+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:27.162 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:27.160+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:28.131 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:28.130+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:28.149 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:28.147+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:29.090 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:29.088+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:29.153 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:29.151+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:30.132 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:30.131+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:30.169 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:30.167+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:31.157 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:31.155+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:31.211 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:31.176+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:32.133 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:32.131+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:32.201 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:32.199+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:33.139 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:33.137+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:33.210 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:33.208+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:34.120 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:34.118+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:34.168 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:34.166+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:35.071 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:35.069+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:35.168 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:35.166+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:36.060 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:36.058+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:36.161 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:36.160+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:37.031 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:37.030+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:37.148 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:37.147+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:38.077 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:38.076+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:38.144 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:38.143+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:39.099 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:39.098+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:39.183 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:39.182+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:40.094 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:40.093+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:40.147 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:40.146+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:41.084 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:41.083+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:41.194 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:41.194+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:42.073 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:42.072+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:42.237 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:42.237+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:43.051 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:43.050+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:43.217 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:43.216+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:44.099 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:44.098+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:44.214 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:44.210+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:45.056 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:45.056+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:45.191 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:45.190+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:46.022 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:46.021+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:46.202 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:46.201+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:47.005 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:47.004+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:47.228 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:47.227+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:47.999 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:47.999+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:48.240 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:48.239+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:48.952 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:48.951+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:49.229 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:49.228+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:49.933 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:49.932+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:50.222 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:50.222+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:50.979 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:50.978+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:51.235 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:51.235+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:51.995 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:51.995+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:52.255 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:52.255+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:53.027 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:53.026+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:53.255 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:53.255+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:54.037 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:54.036+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:54.219 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:54.219+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:55.056 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:55.055+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:55.210 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:55.209+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:56.038 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:56.036+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:56.172 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:56.171+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:57.047 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:57.045+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:57.135 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:57.134+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:58.001 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:57.999+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:58.128 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:58.126+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:58.952 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:58.951+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:59.091 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:49:59.089+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:49:59.924 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:49:59.922+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:00.104 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:00.103+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:00.936 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:00.935+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:01.141 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:01.139+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:01.982 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:01.980+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:02.120 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:02.118+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:02.959 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:02.958+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:03.091 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:03.089+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:04.001 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:04.000+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:04.042 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:04.041+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:04.975 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:04.974+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:05.057 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:05.055+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:05.998 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:05.996+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:06.048 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:06.045+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:06.988 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:06.986+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:07.003 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:07.001+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:07.940 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:07.937+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:08.036 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:08.034+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:08.895 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:08.894+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:09.014 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:09.012+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:09.873 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:09.871+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:09.965 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:09.963+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:10.848 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:10.847+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:10.949 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:10.947+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:11.893 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:11.890+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:11.915 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:11.913+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:12.852 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:12.850+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:12.890 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:12.889+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:13.877 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:13.875+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:13.882 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:13.880+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:14.899 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:14.897+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:14.926 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:14.924+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:15.882 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:15.880+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:15.953 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:15.951+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:16.844 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:16.842+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:16.919 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:16.917+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:17.886 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:17.885+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:17.890 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:17.888+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:18.854 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:18.853+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:18.888 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:18.887+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:19.841 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:19.839+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:19.851 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:19.849+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:20.820 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:20.818+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:20.867 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:20.865+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:21.798 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:21.796+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:21.841 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:21.839+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:22.802 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:22.800+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:22.829 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:22.828+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:23.774 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:23.772+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:23.792 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:23.791+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:24.773 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:24.772+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:24.788 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:24.786+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:25.811 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:25.809+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:25.824 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:25.823+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:26.850 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:26.849+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:26.860 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:26.859+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:27.834 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:27.833+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:27.838 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:27.837+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:28.847 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:28.846+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:28.885 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:28.885+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:29.823 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:29.823+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:29.933 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:29.933+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:30.817 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:30.817+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:30.954 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:30.953+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:31.824 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:31.823+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:32.000 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:31.999+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:32.799 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:32.798+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:32.967 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:32.966+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:33.808 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:33.807+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:33.980 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:33.979+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:34.784 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:34.783+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:34.943 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:34.943+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:35.834 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:35.834+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:35.944 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:35.943+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:36.804 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:36.803+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:36.990 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:36.989+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:37.846 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:37.845+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:38.040 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:38.040+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:38.856 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:38.856+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:39.027 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:39.026+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:39.893 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:39.892+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:40.034 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:40.033+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:40.859 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:40.858+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:41.050 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:41.049+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:41.846 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:41.846+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:42.095 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:42.094+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:42.893 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:42.892+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:43.063 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:43.063+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:43.938 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:43.937+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:44.089 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:44.089+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:44.909 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:44.908+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:45.046 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:45.045+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:45.949 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:45.948+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:46.068 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:46.066+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:46.954 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:46.952+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:47.063 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:47.062+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:47.910 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:47.909+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:48.047 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:48.046+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:48.931 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:48.930+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:49.084 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:49.082+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:49.920 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:49.918+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:50.086 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:50.084+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:50.959 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:50.957+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:51.096 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:51.094+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:51.968 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:51.966+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:52.128 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:52.127+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:52.941 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:52.940+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:53.111 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:53.109+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:53.917 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:53.915+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:54.162 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:54.160+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:54.937 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:54.936+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:55.191 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:55.189+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:55.986 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:55.984+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:56.162 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:56.160+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:56.996 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:56.995+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:57.209 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:57.207+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:57.999 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:57.997+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:58.230 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:58.227+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:58.952 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:58.950+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:59.186 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:50:59.184+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:50:59.929 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:50:59.927+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:00.192 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:00.190+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:00.882 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:00.880+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:01.183 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:01.181+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:01.891 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:01.890+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:02.213 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:02.211+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:02.886 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:02.884+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:03.235 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:03.233+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:03.921 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:03.919+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:04.242 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:04.240+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:04.936 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:04.934+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:05.263 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:05.261+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:05.974 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:05.972+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:06.288 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:06.286+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:06.937 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:06.936+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:07.314 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:07.313+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:07.970 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:07.968+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:08.350 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:08.348+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:09.021 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:09.019+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:09.379 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:09.378+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:10.034 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:10.032+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:10.410 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:10.408+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:11.052 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:11.026+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:11.427 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:11.426+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:11.991 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:11.990+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:12.425 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:12.424+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:13.030 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:13.028+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:13.463 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:13.461+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:14.000 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:13.999+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:14.508 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:14.507+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:14.973 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:14.971+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:15.495 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:15.493+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:15.999 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:15.998+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:16.495 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:16.493+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:16.971 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:16.969+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:17.540 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:17.538+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:17.997 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:17.996+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:18.546 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:18.545+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:19.000 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:18.999+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:19.570 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:19.569+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:19.963 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:19.963+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:20.617 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:20.616+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:20.957 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:20.956+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:21.627 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:21.626+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:21.940 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:21.940+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:22.650 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:22.649+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:22.939 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:22.938+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:23.662 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:23.662+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:23.904 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:23.904+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:24.640 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:24.639+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:24.941 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:24.940+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:25.601 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:25.600+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:25.945 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:25.944+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:26.638 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:26.638+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:26.923 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:26.923+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:27.619 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:27.618+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:27.957 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:27.956+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:28.571 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:28.570+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:28.997 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:28.996+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:29.526 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:29.525+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:30.007 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:30.006+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:30.542 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:30.541+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:30.970 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:30.969+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:31.575 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:31.575+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:31.994 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:31.994+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:32.599 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:32.598+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:32.995 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:32.994+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:33.592 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:33.592+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:34.034 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:34.033+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:34.641 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:34.641+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:35.071 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:35.070+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:35.608 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:35.607+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:36.028 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:36.027+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:36.646 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:36.644+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:37.075 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:37.074+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:37.627 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:37.625+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:38.064 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:38.062+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:38.631 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:38.630+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:39.074 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:39.072+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:39.616 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:39.615+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:40.056 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:40.054+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:40.603 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:40.601+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:41.038 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:41.036+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:41.594 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:41.592+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:42.085 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:42.084+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:42.635 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:42.633+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:43.108 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:43.107+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:43.593 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:43.592+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:44.062 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:44.061+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:44.606 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:44.605+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:45.031 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:45.029+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:45.613 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:45.611+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:46.071 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:46.070+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:46.634 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:46.632+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:47.051 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:47.050+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:47.670 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:47.669+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:48.020 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:48.019+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:48.694 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:48.693+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:49.049 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:49.047+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:49.711 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:49.709+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:50.028 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:50.027+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:50.724 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:50.722+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:51.056 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:51.054+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:51.687 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:51.685+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:52.073 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:52.071+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:52.708 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:52.706+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:53.049 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:53.047+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:53.726 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:53.725+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:54.078 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:54.076+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:54.705 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:54.703+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:55.113 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:55.112+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:55.665 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:55.664+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:56.116 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:56.115+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:56.629 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:56.627+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:57.099 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:57.097+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:57.609 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:57.607+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:58.058 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:58.057+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:58.596 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:58.594+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:59.102 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:51:59.101+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:51:59.615 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:51:59.614+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:00.134 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:00.132+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:00.654 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:00.653+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:01.098 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:01.096+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:01.619 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:01.617+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:02.140 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:02.139+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:02.644 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:02.642+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:03.152 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:03.151+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:03.671 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:03.669+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:04.178 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:04.177+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:04.713 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:04.712+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:05.156 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:05.154+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:05.764 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:05.762+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:06.171 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:06.170+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:06.807 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:06.806+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:07.212 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:07.211+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:07.788 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:07.786+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:08.180 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:08.179+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:08.815 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:08.814+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:09.146 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:09.145+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:09.768 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:09.767+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:10.181 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:10.181+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:10.776 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:10.775+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:11.222 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:11.222+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:11.802 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:11.801+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:12.205 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:12.205+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:12.801 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:12.801+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:13.242 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:13.241+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:13.824 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:13.823+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:14.227 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:14.226+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:14.777 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:14.776+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:15.208 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:15.207+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:15.738 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:15.737+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:16.174 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:16.173+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:16.737 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:16.736+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:17.163 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:17.162+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:17.712 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:17.711+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:18.132 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:18.131+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:18.698 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:18.697+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:19.157 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:19.156+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:19.733 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:19.732+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:20.206 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:20.205+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:20.704 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:20.704+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:21.226 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:21.225+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:21.686 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:21.685+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:22.264 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:22.263+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:22.641 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:22.641+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:23.243 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:23.242+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:23.689 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:23.688+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:24.263 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:24.262+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:24.731 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:24.730+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:25.232 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:25.229+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:25.725 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:25.725+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:26.250 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:26.250+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:26.685 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:26.684+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:27.252 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:27.250+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:27.698 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:27.696+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:28.291 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:28.289+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:28.744 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:28.742+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:29.323 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:29.321+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:29.739 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:29.738+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:30.302 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:30.300+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:30.758 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:30.756+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:31.262 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:31.260+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:31.729 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:31.727+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:32.290 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:32.289+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:32.688 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:32.686+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:33.256 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:33.254+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:33.671 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:33.670+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:34.256 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:34.254+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:34.696 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:34.694+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:35.269 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:35.268+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:35.647 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:35.645+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:36.251 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:36.249+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:36.631 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:36.629+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:37.269 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:37.267+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:37.585 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:37.583+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:38.259 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:38.258+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:38.589 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:38.587+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:39.260 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:39.259+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:39.595 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:39.594+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:40.284 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:40.282+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:40.583 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:40.581+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:41.293 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:41.292+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:41.555 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:41.553+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:42.311 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:42.309+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:42.510 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:42.509+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:43.322 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:43.320+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:43.547 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:43.545+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:44.367 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:44.366+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:44.575 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:44.574+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:45.399 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:45.397+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:45.624 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:45.623+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:46.403 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:46.402+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:46.640 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:46.638+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:47.431 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:47.429+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:47.602 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:47.600+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:48.417 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:48.415+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:48.648 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:48.647+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:49.450 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:49.448+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:49.602 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:49.600+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:50.405 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:50.403+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:50.565 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:50.564+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:51.420 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:51.418+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:51.532 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:51.530+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:52.431 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:52.429+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:52.576 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:52.574+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:53.385 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:53.383+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:53.547 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:53.546+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:54.367 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:54.365+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:54.529 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:54.527+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:55.345 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:55.344+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:55.570 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:55.568+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:56.321 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:56.319+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:56.602 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:56.600+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:57.313 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:57.312+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:57.645 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:57.644+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:58.295 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:58.294+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:58.626 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:58.624+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:59.248 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:52:59.246+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:52:59.609 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:52:59.608+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:00.211 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:00.210+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:00.621 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:00.620+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:01.250 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:01.249+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:01.608 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:01.608+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:02.227 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:02.226+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:02.650 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:02.650+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:03.242 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:03.242+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:03.609 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:03.608+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:04.206 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:04.205+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:04.589 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:04.588+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:05.172 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:05.170+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:05.542 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:05.541+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:06.221 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:06.220+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:06.519 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:06.519+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:07.219 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:07.218+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:07.480 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:07.480+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:08.230 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:08.230+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:08.435 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:08.434+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:09.199 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:09.198+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:09.474 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:09.474+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:10.219 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:10.218+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:10.440 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:10.440+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:11.235 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:11.235+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:11.441 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:11.441+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:12.190 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:12.189+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:12.424 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:12.424+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:13.220 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:13.220+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:13.410 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:13.409+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:14.213 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:14.212+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:14.426 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:14.425+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:15.250 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:15.249+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:15.425 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:15.424+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:16.278 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:16.278+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:16.452 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:16.451+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:17.304 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:17.304+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:17.466 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:17.464+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:18.257 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:18.256+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:18.507 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:18.505+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:19.232 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:19.230+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:19.467 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:19.465+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:20.203 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:20.201+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:20.477 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:20.475+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:21.229 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:21.227+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:21.501 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:21.499+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:22.224 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:22.223+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:22.466 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:22.465+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:23.189 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:23.187+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:23.464 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:23.463+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:24.209 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:24.207+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:24.432 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:24.430+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:25.164 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:25.162+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:25.418 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:25.417+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:26.169 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:26.167+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:26.415 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:26.414+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:27.185 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:27.181+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:27.419 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:27.417+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:28.151 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:28.149+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:28.464 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:28.462+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:29.184 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:29.183+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:29.450 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:29.449+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:30.145 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:30.143+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:30.500 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:30.498+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:31.129 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:31.127+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:31.524 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:31.523+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:32.124 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:32.122+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:32.549 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:32.548+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:33.172 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:33.171+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:33.526 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:33.524+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:34.135 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:34.134+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:34.510 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:34.508+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:35.106 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:35.104+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:35.547 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:35.546+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:36.068 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:36.066+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:36.569 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:36.567+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:37.046 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:37.044+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:37.521 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:37.520+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:38.015 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:38.014+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:38.521 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:38.519+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:39.060 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:39.058+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:39.543 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:39.541+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:40.036 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:40.034+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:40.568 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:40.567+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:41.085 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:41.083+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:41.542 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:41.540+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:42.041 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:42.039+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:42.538 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:42.536+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:43.071 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:43.070+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:43.549 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:43.547+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:44.030 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:44.029+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:44.572 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:44.571+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:45.015 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:45.013+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:45.602 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:45.600+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:45.986 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:45.984+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:46.637 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:46.635+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:46.958 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:46.957+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:47.616 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:47.615+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:47.980 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:47.978+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:48.574 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:48.572+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:48.980 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:48.979+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:49.542 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:49.541+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:49.934 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:49.932+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:50.507 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:50.506+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:50.937 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:50.936+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:51.514 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:51.514+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:51.985 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:51.984+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:52.529 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:52.528+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:52.981 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:52.980+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:53.524 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:53.523+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:53.955 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:53.955+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:54.509 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:54.508+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:54.928 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:54.927+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:55.500 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:55.499+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:55.918 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:55.917+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:56.540 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:56.539+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:56.929 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:56.929+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:57.562 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:57.562+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:57.965 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:57.964+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:58.585 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:58.585+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:58.957 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:58.956+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:59.591 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:53:59.590+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:53:59.963 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:53:59.963+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:00.608 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:00.607+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:00.942 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:00.941+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:01.648 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:01.647+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:01.946 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:01.945+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:02.635 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:02.634+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:02.957 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:02.956+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:03.671 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:03.670+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:03.969 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:03.968+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:04.622 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:04.622+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:04.953 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:04.953+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:05.579 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:05.579+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:05.999 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:05.998+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:06.570 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:06.569+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:06.997 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:06.997+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:07.573 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:07.573+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:07.963 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:07.963+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:08.545 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:08.543+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:08.933 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:08.932+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:09.594 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:09.592+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:09.977 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:09.976+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:10.580 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:10.578+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:10.965 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:10.964+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:11.580 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:11.578+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:11.991 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:11.990+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:12.596 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:12.594+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:12.970 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:12.968+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:13.629 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:13.627+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:13.994 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:13.992+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:14.642 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:14.641+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:14.970 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:14.968+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:15.628 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:15.626+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:15.953 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:15.951+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:16.622 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:16.620+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:16.905 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:16.903+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:17.617 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:17.616+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:17.869 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:17.867+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:18.654 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:18.652+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:18.852 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:18.850+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:19.623 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:19.621+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:19.888 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:19.886+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:20.610 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:20.609+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:20.933 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:20.932+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:21.657 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:21.656+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:21.889 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:21.888+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:22.659 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:22.658+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:22.913 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:22.912+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:23.670 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:23.668+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:23.955 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:23.953+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:24.660 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:24.658+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:24.976 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:24.975+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:25.629 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:25.627+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:25.998 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:25.997+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:26.588 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:26.586+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:27.028 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:27.027+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:27.637 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:27.635+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:27.994 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:27.992+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:28.677 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:28.676+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:28.971 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:28.970+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:29.712 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:29.711+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:30.002 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:30.000+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:30.692 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:30.690+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:30.985 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:30.983+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:31.692 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:31.690+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:32.023 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:32.021+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:32.667 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:32.665+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:33.052 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:33.050+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:33.717 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:33.716+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:34.099 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:34.098+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:34.749 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:34.747+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:35.069 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:35.067+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:35.735 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:35.733+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:36.109 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:36.107+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:36.768 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:36.767+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:37.117 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:37.115+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:37.745 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:37.743+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:38.157 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:38.155+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:38.709 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:38.707+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:39.183 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:39.182+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:39.742 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:39.740+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:40.191 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:40.190+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:40.732 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:40.730+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:41.148 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:41.146+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:41.732 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:41.732+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:42.120 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:42.119+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:42.737 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:42.737+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:43.147 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:43.147+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:43.750 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:43.750+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:44.106 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:44.105+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:44.774 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:44.774+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:45.089 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:45.089+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:45.756 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:45.756+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:46.136 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:46.136+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:46.753 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:46.752+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:47.170 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:47.169+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:47.731 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:47.731+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:48.212 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:48.212+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:48.699 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:48.698+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:49.163 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:49.163+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:49.694 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:49.693+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:50.118 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:50.117+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:50.654 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:50.653+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:51.117 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:51.116+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:51.646 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:51.645+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:52.086 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:52.085+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:52.655 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:52.654+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:53.130 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:53.129+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:53.675 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:53.675+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:54.167 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:54.166+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:54.704 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:54.704+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:55.212 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:55.211+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:55.732 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:55.731+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:56.251 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:56.250+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:56.712 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:56.711+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:57.300 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:57.299+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:57.680 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:57.679+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:58.267 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:58.267+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:58.652 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:58.651+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:59.317 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:54:59.315+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:54:59.630 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:54:59.628+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:00.276 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:00.274+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:00.668 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:00.667+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:01.262 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:01.260+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:01.663 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:01.661+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:02.262 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:02.260+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:02.678 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:02.677+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:03.280 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:03.278+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:03.686 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:03.684+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:04.234 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:04.232+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:04.705 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:04.702+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:05.279 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:05.278+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:05.671 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:05.670+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:06.271 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:06.269+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:06.645 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:06.643+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:07.239 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:07.238+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:07.606 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:07.604+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:08.240 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:08.238+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:08.559 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:08.557+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:09.229 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:09.228+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:09.565 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:09.563+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:10.268 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:10.267+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:10.612 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:10.611+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:11.273 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:11.271+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:11.566 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:11.565+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:12.269 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:12.267+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:12.604 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:12.603+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:13.268 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:13.266+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:13.569 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:13.568+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:14.296 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:14.294+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:14.552 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:14.551+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:15.260 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:15.258+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:15.570 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:15.568+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:16.309 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:16.307+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:16.596 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:16.594+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:17.282 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:17.281+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:17.590 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:17.588+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:18.250 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:18.249+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:18.561 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:18.559+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:19.291 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:19.289+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:19.540 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:19.538+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:20.314 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:20.312+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:20.518 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:20.516+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:21.328 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:21.327+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:21.549 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:21.547+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:22.345 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:22.344+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:22.597 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:22.596+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:23.342 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:23.340+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:23.558 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:23.556+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:24.383 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:24.381+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:24.564 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:24.562+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:25.367 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:25.366+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:25.591 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:25.590+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:26.368 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:26.366+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:26.576 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:26.574+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:27.405 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:27.403+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:27.601 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:27.599+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:28.358 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:28.356+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:28.626 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:28.625+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:29.367 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:29.365+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:29.623 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:29.622+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:30.331 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:30.330+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:30.604 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:30.603+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:31.353 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:31.351+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:31.580 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:31.578+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:32.382 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:32.381+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:32.597 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:32.596+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:33.363 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:33.363+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:33.593 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:33.592+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:34.328 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:34.327+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:34.568 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:34.567+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:35.344 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:35.343+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:35.531 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:35.531+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:36.313 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:36.312+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:36.569 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:36.568+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:37.293 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:37.293+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:37.606 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:37.606+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:38.277 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:38.276+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:38.570 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:38.569+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:39.321 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:39.320+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:39.588 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:39.587+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:40.346 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:40.346+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:40.584 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:40.583+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:41.378 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:41.377+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:41.595 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:41.594+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:42.381 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:42.380+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:42.587 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:42.587+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:43.334 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:43.334+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:43.630 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:43.630+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:44.295 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:44.295+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:44.590 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:44.589+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:45.264 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:45.263+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:45.566 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:45.566+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:46.250 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:46.250+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:46.600 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:46.600+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:47.265 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:47.264+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:47.600 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:47.599+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:48.279 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:48.279+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:48.622 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:48.622+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:49.267 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:49.266+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:49.630 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:49.630+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:50.278 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:50.277+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:50.654 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:50.653+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:51.298 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:51.296+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:51.625 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:51.624+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:52.269 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:52.267+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:52.619 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:52.618+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:53.238 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:53.236+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:53.634 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:53.632+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:54.281 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:54.280+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:54.616 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:54.615+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:55.240 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:55.239+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:55.615 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:55.614+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:56.202 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:56.201+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:56.594 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:56.592+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:57.242 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:57.241+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:57.552 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:57.551+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:58.284 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:58.283+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:58.547 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:58.545+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:59.253 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:55:59.251+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:55:59.525 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:55:59.524+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:00.296 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:00.295+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:00.553 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:00.552+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:01.332 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:01.330+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:01.547 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:01.545+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:02.330 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:02.328+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:02.564 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:02.563+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:03.354 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:03.352+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:03.613 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:03.611+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:04.372 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:04.371+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:04.658 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:04.657+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:05.405 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:05.403+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:05.690 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:05.689+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:06.391 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:06.389+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:06.717 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:06.716+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:07.388 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:07.386+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:07.670 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:07.668+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:08.414 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:08.412+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:08.670 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:08.669+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:09.462 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:09.461+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:09.658 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:09.657+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:10.511 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:10.510+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:10.672 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:10.670+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:11.498 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:11.497+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:11.639 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:11.637+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:12.533 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:12.531+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:12.685 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:12.683+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:13.490 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:13.488+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:13.681 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:13.679+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:14.487 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:14.485+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:14.680 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:14.678+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:15.514 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:15.513+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:15.639 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:15.638+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:16.541 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:16.539+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:16.685 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:16.684+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:17.554 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:17.553+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:17.733 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:17.732+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:18.599 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:18.598+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:18.701 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:18.700+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:19.648 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:19.646+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:19.742 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:19.740+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:20.647 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:20.645+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:20.766 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:20.765+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:21.612 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:21.611+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:21.721 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:21.719+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:22.565 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:22.564+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:22.699 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:22.698+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:23.532 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:23.531+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:23.726 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:23.726+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:24.521 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:24.521+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:24.768 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:24.767+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:25.502 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:25.502+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:25.797 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:25.796+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:26.530 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:26.529+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:26.812 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:26.811+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:27.580 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:27.580+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:27.823 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:27.822+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:28.613 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:28.613+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:28.831 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:28.830+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:29.582 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:29.581+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:29.783 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:29.782+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:30.581 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:30.581+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:30.811 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:30.811+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:31.628 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:31.627+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:31.815 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:31.814+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:32.641 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:32.640+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:32.791 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:32.791+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:33.639 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:33.639+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:33.778 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:33.778+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:34.660 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:34.659+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:34.734 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:34.734+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:35.673 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:35.673+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:35.766 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:35.766+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:36.720 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:36.719+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:36.759 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:36.759+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:37.756 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:37.756+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:37.787 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:37.787+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:38.730 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:38.729+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:38.837 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:38.837+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:39.686 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:39.686+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:39.822 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:39.822+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:40.678 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:40.676+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:40.822 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:40.820+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:41.685 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:41.681+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:41.785 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:41.784+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:42.725 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:42.723+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:42.779 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:42.778+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:43.765 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:43.763+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:43.828 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:43.826+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:44.799 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:44.798+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:44.846 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:44.844+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:45.761 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:45.759+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:45.847 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:45.846+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:46.789 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:46.787+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:46.875 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:46.873+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:47.804 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:47.802+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:47.919 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:47.918+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:48.824 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:48.822+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:48.873 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:48.871+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:49.797 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:49.796+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:49.875 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:49.874+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:50.821 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:50.820+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:50.836 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:50.835+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:51.803 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:51.802+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:51.807 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:51.805+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:52.785 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:52.783+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:52.803 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:52.802+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:53.750 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:53.748+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:53.815 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:53.813+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:54.782 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:54.780+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:54.864 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:54.862+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:55.810 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:55.808+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:55.861 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:55.860+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:56.797 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:56.796+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:56.883 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:56.882+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:57.790 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:57.788+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:57.858 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:57.857+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:58.781 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:58.780+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:58.896 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:58.895+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:59.773 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:56:59.772+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:56:59.908 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:56:59.906+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:00.750 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:00.748+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:00.919 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:00.917+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:01.707 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:01.705+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:01.916 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:01.914+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:02.720 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:02.718+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:02.962 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:02.960+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:03.723 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:03.721+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:03.978 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:03.977+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:04.728 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:04.726+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:04.995 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:04.994+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:05.736 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:05.735+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:06.019 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:06.017+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:06.777 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:06.775+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:07.015 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:07.014+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:07.741 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:07.740+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:08.030 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:08.028+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:08.774 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:08.772+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:08.998 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:08.996+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:09.767 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:09.766+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:09.977 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:09.975+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:10.800 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:10.799+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:10.981 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:10.980+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:11.832 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:11.830+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:12.019 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:12.018+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:12.871 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:12.869+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:12.997 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:12.996+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:13.905 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:13.904+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:14.013 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:14.013+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:14.896 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:14.895+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:15.034 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:15.034+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:15.945 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:15.945+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:16.047 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:16.047+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:16.958 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:16.957+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:17.041 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:17.041+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:17.968 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:17.967+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:18.082 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:18.081+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:18.995 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:18.994+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:19.083 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:19.082+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:20.020 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:20.019+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:20.046 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:20.045+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:21.005 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:21.005+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:21.053 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:21.052+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:22.023 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:22.023+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:22.042 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:22.042+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:22.977 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:22.977+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:23.007 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:23.006+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:23.932 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:23.931+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:24.004 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:24.004+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:24.917 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:24.916+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:24.960 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:24.959+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:25.943 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:25.943+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:25.979 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:25.978+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:26.918 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:26.918+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:26.945 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:26.944+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:27.896 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:27.896+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:27.961 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:27.961+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:28.937 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:28.937+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:28.984 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:28.984+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:29.973 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:29.969+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:29.983 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:29.982+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:30.947 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:30.947+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:30.953 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:30.953+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:31.907 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:31.905+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:31.949 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:31.948+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:32.923 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:32.921+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:32.924 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:32.923+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:33.900 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:33.898+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:33.975 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:33.974+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:34.937 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:34.935+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:35.000 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:34.999+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:35.917 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:35.915+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:36.047 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:36.046+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:36.922 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:36.921+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:37.013 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:37.012+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:37.909 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:37.907+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:38.014 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:38.013+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:38.957 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:38.955+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:38.995 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:38.993+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:39.986 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:39.984+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:40.030 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:40.028+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:41.002 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:41.001+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:41.033 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:41.031+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:41.965 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:41.964+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:42.082 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:42.081+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:42.993 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:42.992+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:43.040 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:43.038+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:43.991 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:43.989+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:43.994 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:43.992+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:44.945 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:44.944+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:45.041 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:45.039+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:45.905 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:45.904+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:46.034 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:46.033+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:46.897 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:46.895+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:47.046 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:47.044+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:47.893 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:47.892+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:48.027 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:48.025+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:48.902 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:48.900+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:49.028 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:49.026+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:49.945 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:49.944+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:50.071 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:50.069+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:50.976 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:50.974+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:51.071 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:51.070+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:51.997 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:51.995+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:52.101 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:52.100+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:52.949 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:52.947+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:53.145 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:53.144+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:53.970 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:53.969+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:54.170 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:54.168+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:54.947 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:54.946+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:55.178 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:55.176+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:55.922 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:55.921+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:56.177 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:56.175+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:56.967 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:56.965+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:57.183 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:57.181+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:57.973 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:57.972+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:58.136 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:58.134+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:58.939 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:58.937+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:59.148 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:57:59.146+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:57:59.932 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:57:59.930+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:00.148 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:00.146+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:00.943 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:00.942+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:01.147 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:01.146+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:01.959 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:01.958+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:02.176 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:02.175+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:02.977 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:02.976+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:03.127 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:03.126+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:03.941 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:03.940+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:04.176 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:04.175+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:04.900 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:04.899+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:05.214 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:05.214+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:05.943 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:05.943+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:06.254 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:06.254+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:06.957 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:06.956+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:07.228 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:07.227+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:07.915 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:07.915+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:08.205 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:08.205+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:08.935 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:08.934+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:09.171 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:09.171+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:09.930 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:09.929+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:10.128 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:10.128+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:10.905 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:10.904+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:11.135 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:11.134+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:11.890 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:11.889+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:12.165 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:12.165+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:12.891 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:12.890+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:13.216 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:13.216+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:13.883 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:13.882+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:14.240 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:14.239+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:14.834 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:14.834+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:15.271 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:15.271+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:15.868 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:15.868+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:16.248 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:16.248+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:16.893 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:16.893+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:17.256 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:17.255+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:17.851 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:17.850+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:18.213 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:18.213+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:18.886 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:18.885+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:19.206 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:19.206+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:19.837 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:19.837+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:20.167 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:20.167+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:20.844 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:20.843+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:21.134 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:21.134+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:21.842 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:21.841+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:22.107 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:22.106+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:22.846 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:22.845+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:23.150 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:23.149+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:23.802 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:23.800+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:24.164 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:24.163+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:24.837 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:24.835+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:25.197 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:25.196+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:25.824 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:25.823+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:26.171 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:26.169+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:26.805 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:26.804+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:27.211 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:27.209+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:27.801 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:27.800+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:28.173 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:28.169+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:28.764 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:28.762+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:29.149 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:29.147+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:29.735 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:29.733+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:30.197 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:30.195+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:30.745 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:30.744+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:31.206 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:31.205+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:31.781 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:31.780+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:32.212 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:32.210+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:32.852 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:32.829+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:33.241 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:33.240+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:33.787 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:33.785+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:34.279 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:34.277+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:34.742 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:34.741+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:35.305 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:35.303+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:35.716 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:35.715+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:36.274 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:36.272+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:36.730 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:36.728+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:37.307 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:37.306+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:37.701 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:37.699+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:38.342 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:38.341+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:38.670 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:38.668+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:39.389 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:39.388+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:39.626 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:39.625+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:40.364 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:40.362+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:40.589 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:40.587+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:41.357 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:41.355+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:41.565 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:41.563+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:42.373 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:42.372+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:42.577 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:42.575+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:43.327 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:43.325+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:43.598 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:43.597+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:44.308 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:44.306+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:44.584 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:44.583+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:45.319 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:45.318+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:45.609 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:45.608+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:46.321 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:46.320+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:46.574 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:46.572+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:47.370 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:47.368+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:47.543 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:47.542+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:48.348 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:48.347+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:48.582 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:48.581+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:49.330 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:49.328+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:49.631 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:49.630+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:50.325 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:50.324+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:50.603 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:50.602+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:51.359 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:51.358+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:51.640 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:51.639+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:52.380 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:52.379+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:52.599 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:52.598+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:53.397 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:53.395+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:53.638 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:53.636+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:54.441 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:54.440+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:54.665 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:54.664+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:55.423 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:55.422+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:55.699 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:55.698+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:56.411 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:56.410+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:56.683 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:56.682+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:57.457 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:57.456+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:57.667 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:57.667+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:58.435 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:58.423+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:58.692 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:58.691+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:59.438 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:58:59.437+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:58:59.726 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:58:59.726+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:00.468 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:00.467+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:00.677 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:00.677+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:01.474 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:01.474+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:01.717 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:01.717+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:02.508 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:02.507+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:02.736 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:02.736+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:03.529 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:03.528+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:03.698 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:03.697+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:04.571 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:04.571+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:04.661 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:04.661+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:05.533 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:05.533+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:05.613 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:05.612+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:06.583 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:06.583+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:06.620 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:06.619+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:07.600 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:07.599+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:07.610 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:07.610+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:08.622 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:08.622+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:08.649 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:08.649+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:09.626 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:09.626+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:09.649 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:09.648+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:10.634 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:10.633+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:10.691 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:10.690+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:11.598 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:11.597+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:11.701 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:11.700+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:12.612 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:12.612+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:12.732 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:12.731+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:13.641 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:13.639+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:13.781 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:13.780+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:14.658 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:14.656+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:14.779 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:14.778+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:15.663 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:15.662+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:15.753 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:15.751+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:16.701 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:16.700+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:16.704 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:16.703+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:17.679 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:17.677+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:17.729 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:17.727+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:18.639 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:18.637+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:18.752 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:18.751+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:19.616 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:19.615+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:19.739 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:19.737+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:20.634 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:20.633+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:20.698 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:20.696+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:21.641 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:21.639+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:21.696 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:21.694+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:22.674 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:22.672+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:22.686 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:22.684+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:23.670 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:23.668+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:23.727 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:23.726+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:24.719 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:24.718+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:24.720 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:24.719+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:25.675 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:25.674+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:25.736 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:25.735+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:26.630 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:26.628+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:26.690 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:26.688+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:27.583 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:27.582+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:27.707 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:27.705+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:28.627 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:28.625+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:28.692 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:28.691+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:29.590 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:29.588+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:29.645 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:29.644+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:30.585 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:30.583+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:30.682 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:30.680+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:31.612 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:31.611+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:31.689 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:31.687+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:32.642 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:32.641+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:32.674 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:32.673+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:33.631 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:33.629+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:33.639 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:33.638+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:34.583 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:34.581+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:34.594 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:34.592+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:35.619 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:35.617+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:35.629 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:35.628+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:36.575 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:36.573+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:36.628 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:36.627+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:37.551 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:37.550+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:37.655 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:37.653+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:38.551 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:38.550+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:38.641 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:38.640+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:39.551 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:39.549+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:39.681 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:39.680+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:40.520 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:40.519+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:40.700 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:40.698+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:41.568 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:41.567+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:41.690 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:41.688+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:42.582 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:42.580+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:42.660 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:42.658+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:43.549 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:43.548+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:43.658 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:43.657+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:44.569 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:44.567+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:44.663 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:44.662+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:45.599 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:45.598+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:45.659 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:45.658+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:46.614 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:46.613+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:46.632 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:46.632+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:47.623 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:47.622+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:47.671 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:47.671+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:48.638 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:48.638+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:48.662 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:48.662+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:49.629 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:49.628+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:49.658 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:49.657+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:50.630 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:50.630+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:50.641 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:50.641+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:51.656 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:51.655+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:51.657 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:51.657+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:52.651 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:52.651+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:52.659 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:52.659+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:53.666 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:53.666+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:53.683 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:53.683+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:54.653 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:54.653+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:54.712 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:54.712+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:55.669 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:55.669+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:55.674 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:55.674+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:56.655 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:56.655+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:56.713 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:56.712+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:57.642 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:57.642+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:57.673 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:57.672+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:58.673 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:58.672+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:58.673 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:58.672+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:59.627 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T17:59:59.627+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T17:59:59.710 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T17:59:59.709+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:00.622 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:00.622+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:00.702 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:00.701+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:01.624 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:01.624+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:01.721 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:01.720+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:02.623 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:02.623+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:02.766 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:02.765+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:03.635 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:03.635+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:03.738 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:03.736+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:04.602 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:04.601+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:04.734 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:04.733+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:05.560 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:05.559+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:05.780 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:05.778+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:06.515 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:06.513+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:06.731 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:06.730+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:07.481 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:07.480+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:07.682 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:07.681+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:08.490 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:08.489+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:08.730 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:08.729+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:09.498 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:09.496+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:09.730 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:09.729+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:10.487 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:10.486+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:10.736 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:10.734+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:11.493 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:11.492+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:11.763 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:11.761+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:12.472 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:12.471+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:12.723 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:12.722+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:13.473 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:13.471+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:13.732 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:13.730+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:14.491 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:14.489+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:14.774 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:14.773+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:15.458 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:15.456+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:15.806 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:15.804+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:16.463 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:16.462+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:16.791 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:16.790+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:17.470 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:17.468+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:17.815 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:17.814+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:18.470 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:18.469+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:18.812 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:18.811+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:19.455 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:19.453+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:19.812 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:19.810+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:20.442 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:20.441+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:20.795 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:20.793+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:21.470 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:21.468+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:21.781 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:21.779+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:22.487 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:22.486+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:22.739 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:22.737+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:23.484 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:23.483+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:23.713 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:23.712+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:24.483 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:24.481+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:24.716 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:24.715+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:25.468 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:25.466+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:25.734 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:25.732+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:26.436 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:26.434+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:26.753 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:26.752+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:27.391 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:27.390+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:27.738 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:27.736+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:28.442 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:28.441+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:28.736 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:28.735+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:29.480 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:29.479+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:29.753 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:29.751+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:30.487 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:30.486+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:30.777 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:30.775+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:31.512 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:31.511+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:31.816 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:31.815+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:32.513 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:32.512+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:32.802 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:32.801+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:33.490 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:33.489+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:33.830 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:33.829+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:34.483 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:34.482+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:34.855 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:34.854+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:35.449 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:35.448+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:35.901 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:35.899+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:36.437 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:36.436+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:36.944 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:36.944+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:37.469 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:37.468+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:37.928 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:37.927+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:38.436 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:38.435+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:38.896 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:38.895+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:39.433 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:39.433+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:39.859 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:39.858+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:40.452 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:40.452+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:40.873 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:40.872+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:41.461 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:41.461+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:41.860 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:41.860+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:42.487 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:42.486+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:42.878 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:42.878+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:43.444 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:43.443+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:43.854 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:43.854+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:44.405 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:44.405+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:44.878 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:44.878+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:45.360 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:45.359+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:45.862 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:45.862+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:46.353 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:46.352+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:46.855 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:46.854+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:47.377 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:47.377+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:47.807 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:47.806+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:48.399 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:48.399+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:48.808 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:48.808+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:49.430 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:49.430+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:49.807 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:49.807+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:50.388 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:50.388+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:50.778 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:50.777+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:51.407 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:51.406+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:51.737 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:51.737+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:52.438 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:52.438+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:52.718 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:52.717+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:53.477 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:53.477+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:53.741 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:53.740+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:54.510 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:54.509+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:54.782 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:54.781+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:55.488 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:55.487+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:55.800 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:55.799+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:56.462 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:56.461+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:56.817 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:56.816+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:57.457 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:57.455+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:57.773 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:57.772+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:58.419 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:58.417+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:58.816 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:58.814+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:59.383 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:00:59.381+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:00:59.829 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:00:59.827+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:00.421 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:00.419+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:00.847 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:00.846+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:01.422 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:01.421+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:01.815 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:01.814+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:02.459 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:02.457+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:02.837 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:02.835+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:03.483 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:03.481+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:03.844 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:03.843+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:04.477 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:04.475+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:04.821 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:04.819+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:05.440 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:05.438+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:05.852 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:05.850+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:06.474 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:06.472+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:06.884 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:06.883+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:07.433 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:07.432+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:07.908 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:07.907+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:08.392 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:08.390+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:08.894 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:08.892+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:09.359 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:09.357+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:09.864 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:09.862+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:10.316 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:10.314+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:10.832 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:10.831+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:11.355 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:11.353+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:11.812 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:11.810+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:12.385 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:12.384+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:12.850 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:12.849+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:13.397 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:13.396+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:13.889 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:13.888+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:14.354 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:14.353+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:14.916 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:14.915+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:15.326 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:15.324+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:15.880 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:15.879+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:16.354 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:16.352+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:16.870 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:16.869+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:17.398 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:17.396+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:17.915 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:17.914+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:18.420 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:18.418+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:18.890 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:18.888+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:19.424 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:19.423+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:19.937 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:19.936+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:20.410 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:20.409+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:20.964 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:20.962+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:21.424 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:21.423+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:21.979 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:21.977+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:22.423 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:22.421+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:22.985 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:22.983+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:23.376 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:23.375+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:23.991 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:23.990+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:24.394 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:24.393+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:25.039 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:25.038+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:25.347 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:25.346+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:26.034 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:26.032+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:26.345 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:26.344+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:27.073 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:27.073+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:27.307 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:27.306+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:28.126 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:28.123+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:28.286 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:28.285+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:29.075 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:29.074+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:29.326 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:29.326+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:30.026 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:30.025+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:30.361 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:30.360+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:31.025 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:31.025+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:31.319 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:31.319+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:32.029 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:32.029+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:32.355 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:32.355+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:32.985 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:32.984+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:33.383 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:33.383+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:34.012 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:34.012+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:34.384 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:34.384+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:34.999 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:34.999+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:35.381 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:35.380+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:35.991 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:35.991+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:36.386 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:36.385+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:36.961 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:36.961+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:37.359 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:37.359+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:37.922 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:37.922+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:38.374 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:38.373+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:38.971 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:38.971+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:39.383 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:39.383+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:40.022 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:40.021+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:40.387 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:40.387+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:41.043 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:41.043+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:41.348 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:41.347+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:41.999 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:41.999+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:42.339 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:42.338+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:43.020 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:43.020+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:43.327 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:43.327+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:43.975 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:43.975+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:44.346 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:44.346+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:44.965 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:44.964+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:45.334 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:45.332+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:45.932 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:45.930+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:46.300 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:46.299+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:46.887 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:46.885+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:47.334 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:47.332+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:47.854 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:47.852+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:48.376 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:48.375+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:48.834 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:48.832+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:49.354 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:49.352+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:49.793 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:49.792+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:50.374 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:50.373+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:50.837 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:50.835+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:51.385 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:51.383+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:51.859 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:51.858+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:52.417 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:52.415+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:52.885 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:52.883+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:53.464 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:53.463+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:53.874 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:53.872+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:54.425 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:54.424+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:54.862 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:54.860+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:55.419 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:55.418+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:55.889 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:55.888+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:56.465 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:56.464+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:56.851 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:56.850+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:57.454 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:57.453+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:57.846 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:57.844+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:58.450 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:58.449+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:58.871 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:58.870+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:59.491 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:01:59.490+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:01:59.910 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:01:59.909+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:00.506 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:00.504+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:00.869 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:00.867+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:01.533 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:01.531+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:01.834 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:01.833+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:02.535 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:02.533+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:02.852 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:02.850+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:03.586 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:03.584+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:03.845 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:03.844+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:04.556 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:04.555+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:04.895 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:04.894+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:05.523 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:05.522+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:05.939 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:05.938+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:06.491 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:06.490+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:06.897 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:06.895+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:07.448 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:07.447+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:07.927 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:07.925+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:08.440 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:08.439+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:08.886 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:08.885+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:09.407 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:09.406+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:09.853 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:09.851+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:10.457 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:10.456+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:10.831 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:10.829+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:11.484 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:11.483+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:11.835 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:11.834+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:12.517 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:12.516+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:12.791 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:12.789+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:13.487 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:13.485+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:13.749 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:13.747+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:14.466 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:14.465+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:14.778 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:14.776+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:15.497 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:15.495+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:15.739 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:15.737+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:16.498 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:16.497+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:16.751 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:16.749+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:17.506 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:17.504+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:17.726 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:17.725+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:18.520 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:18.519+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:18.765 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:18.765+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:19.544 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:19.544+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:19.796 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:19.795+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:20.560 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:20.560+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:20.749 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:20.749+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:21.590 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:21.590+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:21.798 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:21.798+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:22.594 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:22.594+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:22.817 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:22.816+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:23.572 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:23.572+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:23.826 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:23.826+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:24.557 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:24.557+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:24.792 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:24.792+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:25.586 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:25.586+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:25.752 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:25.752+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:26.555 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:26.555+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:26.712 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:26.711+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:27.546 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:27.546+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:27.690 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:27.689+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:28.547 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:28.547+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:28.667 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:28.666+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:29.526 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:29.526+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:29.710 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:29.710+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:30.497 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:30.496+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:30.700 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:30.700+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:31.540 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:31.539+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:31.748 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:31.748+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:32.493 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:32.493+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:32.789 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:32.789+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:33.516 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:33.515+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:33.834 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:33.834+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:34.474 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:34.473+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:34.790 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:34.790+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:35.451 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:35.451+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:35.786 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:35.786+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:36.490 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:36.488+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:36.753 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:36.752+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:37.466 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:37.465+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:37.760 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:37.758+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:38.465 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:38.463+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:38.760 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:38.758+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:39.469 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:39.468+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:39.761 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:39.759+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:40.467 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:40.466+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:40.765 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:40.764+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:41.435 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:41.434+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:41.785 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:41.784+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:42.394 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:42.393+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:42.814 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:42.812+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:43.412 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:43.410+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:43.831 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:43.829+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:44.431 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:44.430+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:44.859 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:44.858+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:45.415 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:45.414+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:45.817 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:45.816+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:46.404 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:46.403+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:46.802 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:46.801+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:47.429 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:47.427+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:47.814 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:47.813+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:48.459 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:48.458+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:48.817 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:48.816+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:49.480 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:49.478+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:49.849 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:49.847+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:50.472 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:50.471+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:50.897 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:50.895+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:51.517 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:51.516+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:51.896 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:51.895+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:52.550 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:52.549+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:52.880 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:52.879+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:53.546 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:53.545+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:53.859 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:53.858+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:54.510 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:54.509+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:54.832 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:54.831+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:55.480 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:55.479+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:55.804 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:55.802+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:56.500 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:56.499+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:56.823 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:56.822+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:57.490 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:57.489+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:57.840 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:57.839+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:58.471 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:58.470+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:58.794 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:58.792+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:59.517 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:02:59.516+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:02:59.762 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:02:59.761+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:00.540 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:00.538+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:00.807 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:00.805+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:01.565 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:01.564+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:01.790 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:01.789+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:02.596 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:02.594+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:02.778 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:02.777+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:03.562 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:03.560+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:03.788 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:03.787+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:04.521 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:04.519+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:04.796 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:04.795+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:05.563 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:05.562+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:05.845 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:05.843+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:06.558 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:06.557+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:06.845 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:06.844+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:07.589 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:07.588+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:07.869 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:07.868+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:08.584 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:08.582+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:08.904 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:08.903+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:09.146 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T18:03:09.146+0000 7fd64ac56700 0 -- 172.21.15.82:0/3601686610 >> [v2:172.21.15.82:6808/87261,v1:172.21.15.82:6809/87261] conn(0x7fd630030040 msgr2=0x7fd630065fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-18T18:03:09.626 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:09.625+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:09.907 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:09.856+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:10.594 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:10.594+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:10.836 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:10.835+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:11.622 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:11.621+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:11.879 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:11.879+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:12.606 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:12.606+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:12.890 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:12.890+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:13.654 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:13.654+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:13.892 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:13.892+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:14.607 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:14.607+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:14.877 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:14.876+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:15.572 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:15.572+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:15.872 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:15.872+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:16.530 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:16.529+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:16.838 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:16.837+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:17.558 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:17.558+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:17.852 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:17.852+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:18.546 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:18.546+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:18.883 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:18.882+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:19.591 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:19.591+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:19.927 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:19.926+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:20.612 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:20.611+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:20.898 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:20.898+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:21.628 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:21.628+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:21.889 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:21.888+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:22.619 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:22.618+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:22.900 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:22.899+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:23.668 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:23.668+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:23.928 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:23.928+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:24.652 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:24.652+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:24.960 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:24.960+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:25.666 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:25.666+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:25.964 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:25.964+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:26.619 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:26.619+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:26.989 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:26.987+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:27.591 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:27.589+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:28.014 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:28.012+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:28.606 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:28.604+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:29.013 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:29.012+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:29.590 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:29.588+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:29.992 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:29.991+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:30.610 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:30.609+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:31.027 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:31.025+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:31.644 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:31.642+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:32.053 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:32.052+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:32.690 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:32.688+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:33.051 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:33.050+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:33.652 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:33.651+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:34.043 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:34.042+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:34.690 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:34.689+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:35.083 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:35.081+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:35.682 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:35.681+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:36.078 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:36.076+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:36.701 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:36.699+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:37.106 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:37.104+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:37.734 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:37.733+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:38.089 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:38.087+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:38.691 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:38.689+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:39.076 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:39.074+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:39.695 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:39.693+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:40.078 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:40.076+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:40.742 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:40.741+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:41.041 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:41.039+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:41.772 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:41.770+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:42.019 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:42.018+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:42.734 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:42.733+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:42.991 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:42.990+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:43.740 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:43.739+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:44.002 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:44.001+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:44.784 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:44.783+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:45.040 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:45.039+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:45.738 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:45.737+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:46.078 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:46.077+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:46.710 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:46.709+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:47.103 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:47.102+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:47.696 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:47.694+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:48.083 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:48.081+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:48.679 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:48.677+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:49.039 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:49.038+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:49.662 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:49.660+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:50.086 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:50.085+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:50.626 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:50.624+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:51.046 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:51.045+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:51.636 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:51.635+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:52.058 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:52.057+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:52.677 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:52.675+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:53.032 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:53.030+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:53.692 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:53.690+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:54.031 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:54.030+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:54.679 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:54.677+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:55.036 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:55.035+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:55.660 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:55.659+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:56.066 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:56.065+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:56.703 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:56.702+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:57.069 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:57.067+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:57.694 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:57.693+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:58.053 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:58.052+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:58.680 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:58.679+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:59.103 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:03:59.102+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:03:59.669 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:03:59.667+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:00.078 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:00.078+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:00.650 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:00.650+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:01.106 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:01.106+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:01.608 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:01.607+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:02.104 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:02.104+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:02.653 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:02.652+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:03.064 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:03.064+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:03.703 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:03.702+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:04.085 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:04.084+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:04.698 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:04.698+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:05.125 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:05.125+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:05.655 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:05.654+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:06.096 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:06.096+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:06.654 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:06.654+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:07.082 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:07.081+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:07.623 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:07.623+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:08.075 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:08.074+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:08.576 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:08.576+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:09.084 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:09.083+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:09.609 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:09.609+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:10.092 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:10.091+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:10.649 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:10.649+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:11.121 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:11.121+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:11.631 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:11.631+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:12.075 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:12.075+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:12.655 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:12.655+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:13.084 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:13.084+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:13.655 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:13.655+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:14.070 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:14.070+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:14.693 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:14.693+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:15.097 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:15.097+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:15.734 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:15.733+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:16.051 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:16.051+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:16.766 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:16.766+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:17.014 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:17.013+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:17.758 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:17.757+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:17.995 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:17.994+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:18.770 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:18.769+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:18.992 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:18.991+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:19.765 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:19.764+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:20.026 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:20.025+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:20.768 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:20.766+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:21.025 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:21.023+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:21.739 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:21.737+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:22.045 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:22.044+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:22.766 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:22.765+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:23.022 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:23.020+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:23.735 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:23.734+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:23.987 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:23.985+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:24.729 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:24.728+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:25.019 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:25.018+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:25.714 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:25.713+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:26.037 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:26.035+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:26.735 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:26.734+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:26.994 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:26.993+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:27.785 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:27.783+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:28.000 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:27.998+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:28.806 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:28.804+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:29.012 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:29.010+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:29.770 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:29.769+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:30.030 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:30.029+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:30.780 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:30.779+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:31.002 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:31.001+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:31.735 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:31.733+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:31.974 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:31.973+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:32.735 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:32.734+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:32.953 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:32.951+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:33.725 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:33.724+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:33.975 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:33.973+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:34.750 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:34.748+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:34.986 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:34.985+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:35.800 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:35.799+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:35.940 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:35.939+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:36.800 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:36.798+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:36.892 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:36.890+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:37.801 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:37.800+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:37.873 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:37.872+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:38.757 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:38.756+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:38.891 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:38.890+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:39.777 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:39.776+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:39.877 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:39.875+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:40.795 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:40.794+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:40.874 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:40.873+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:41.767 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:41.765+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:41.877 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:41.875+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:42.726 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:42.724+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:42.882 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:42.880+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:43.752 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:43.751+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:43.851 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:43.849+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:44.746 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:44.744+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:44.816 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:44.815+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:45.713 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:45.711+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:45.775 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:45.773+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:46.717 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:46.715+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:46.814 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:46.813+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:47.737 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:47.736+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:47.817 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:47.815+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:48.689 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:48.688+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:48.789 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:48.787+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:49.728 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:49.726+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:49.749 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:49.747+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:50.722 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:50.722+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:50.727 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:50.726+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:51.697 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:51.697+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:51.754 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:51.753+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:52.700 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:52.699+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:52.757 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:52.757+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:53.716 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:53.715+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:53.731 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:53.731+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:54.688 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:54.687+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:54.740 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:54.740+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:55.650 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:55.650+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:55.695 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:55.694+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:56.626 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:56.626+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:56.683 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:56.683+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:57.656 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:57.656+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:57.720 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:57.720+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:58.630 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:58.629+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:58.729 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:58.729+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:59.668 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:04:59.668+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:04:59.758 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:04:59.758+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:00.685 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:00.684+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:00.734 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:00.733+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:01.696 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:01.695+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:01.703 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:01.703+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:02.658 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:02.658+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:02.690 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:02.690+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:03.688 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:03.687+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:03.691 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:03.691+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:04.704 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:04.704+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:04.705 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:04.705+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:05.690 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:05.690+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:05.714 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:05.714+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:06.652 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:06.652+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:06.686 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:06.685+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:07.648 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:07.647+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:07.702 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:07.702+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:08.617 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:08.616+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:08.746 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:08.745+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:09.627 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:09.626+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:09.697 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:09.696+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:10.656 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:10.655+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:10.717 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:10.716+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:11.676 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:11.675+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:11.681 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:11.680+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:12.654 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:12.652+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:12.665 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:12.663+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:13.664 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:13.663+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:13.678 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:13.676+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:14.700 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:14.698+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:14.713 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:14.711+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:15.683 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:15.682+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:15.696 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:15.695+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:16.644 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:16.643+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:16.719 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:16.717+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:17.599 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:17.597+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:17.744 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:17.743+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:18.620 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:18.619+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:18.774 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:18.773+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:19.643 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:19.642+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:19.768 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:19.767+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:20.604 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:20.603+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:20.782 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:20.781+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:21.626 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:21.624+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:21.744 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:21.743+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:22.659 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:22.658+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:22.742 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:22.741+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:23.672 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:23.671+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:23.779 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:23.777+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:24.643 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:24.641+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:24.757 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:24.756+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:25.652 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:25.651+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:25.801 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:25.800+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:26.604 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:26.603+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:26.842 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:26.841+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:27.655 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:27.653+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:27.882 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:27.880+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:28.612 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:28.611+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:28.892 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:28.891+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:29.607 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:29.605+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:29.909 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:29.907+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:30.650 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:30.649+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:30.936 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:30.935+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:31.689 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:31.687+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:31.905 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:31.904+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:32.736 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:32.735+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:32.916 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:32.915+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:33.699 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:33.698+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:33.904 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:33.903+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:34.745 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:34.744+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:34.903 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:34.901+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:35.710 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:35.708+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:35.862 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:35.861+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:36.723 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:36.722+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:36.881 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:36.880+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:37.688 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:37.686+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:37.842 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:37.841+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:38.702 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:38.701+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:38.825 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:38.824+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:39.698 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:39.696+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:39.789 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:39.788+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:40.744 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:40.743+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:40.786 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:40.784+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:41.777 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:41.777+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:41.791 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:41.791+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:42.744 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:42.744+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:42.763 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:42.763+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:43.723 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:43.723+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:43.765 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:43.765+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:44.717 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:44.717+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:44.757 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:44.756+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:45.761 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:45.761+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:45.778 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:45.778+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:46.753 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:46.753+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:46.792 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:46.792+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:47.704 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:47.703+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:47.826 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:47.825+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:48.722 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:48.722+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:48.845 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:48.845+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:49.720 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:49.720+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:49.883 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:49.883+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:50.738 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:50.738+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:50.883 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:50.882+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:51.747 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:51.746+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:51.858 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:51.857+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:52.767 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:52.766+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:52.863 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:52.862+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:53.761 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:53.761+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:53.855 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:53.855+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:54.729 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:54.729+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:54.836 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:54.835+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:55.703 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:55.702+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:55.859 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:55.858+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:56.731 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:56.731+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:56.834 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:56.834+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:57.705 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:57.704+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:57.799 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:57.798+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:58.702 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:58.700+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:58.771 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:58.770+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:59.707 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:05:59.706+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:05:59.736 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:05:59.734+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:00.738 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:00.737+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:00.755 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:00.754+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:01.757 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:01.756+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:01.780 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:01.778+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:02.711 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:02.709+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:02.741 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:02.739+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:03.711 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:03.710+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:03.785 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:03.783+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:04.694 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:04.692+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:04.829 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:04.828+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:05.724 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:05.723+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:05.816 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:05.815+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:06.761 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:06.760+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:06.774 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:06.773+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:07.778 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:07.777+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:07.795 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:07.794+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:08.783 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:08.781+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:08.786 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:08.785+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:09.770 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:09.768+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:09.803 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:09.801+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:10.743 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:10.742+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:10.774 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:10.773+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:11.787 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:11.786+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:11.795 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:11.794+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:12.769 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:12.768+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:12.810 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:12.809+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:13.772 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:13.771+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:13.857 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:13.856+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:14.821 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:14.820+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:14.860 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:14.859+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:15.806 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:15.805+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:15.909 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:15.908+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:16.789 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:16.788+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:16.904 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:16.903+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:17.760 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:17.759+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:17.926 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:17.925+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:18.797 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:18.796+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:18.910 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:18.908+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:19.824 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:19.823+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:19.933 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:19.932+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:20.865 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:20.864+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:20.919 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:20.917+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:21.905 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:21.903+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:21.915 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:21.914+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:22.910 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:22.908+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:22.946 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:22.945+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:23.921 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:23.920+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:23.955 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:23.954+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:24.915 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:24.914+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:24.983 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:24.982+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:25.871 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:25.870+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:25.971 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:25.970+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:26.884 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:26.882+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:26.938 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:26.936+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:27.846 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:27.845+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:27.949 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:27.947+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:28.839 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:28.838+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:28.901 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:28.900+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:29.872 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:29.871+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:29.878 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:29.878+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:30.836 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:30.834+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:30.894 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:30.893+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:31.859 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:31.858+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:31.871 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:31.870+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:32.899 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:32.898+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:32.914 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:32.913+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:33.885 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:33.884+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:33.917 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:33.916+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:34.889 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:34.888+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:34.933 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:34.933+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:35.924 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:35.923+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:35.957 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:35.956+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:36.897 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:36.897+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:36.983 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:36.983+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:37.855 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:37.855+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:37.945 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:37.944+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:38.868 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:38.868+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:38.942 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:38.942+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:39.847 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:39.846+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:39.919 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:39.919+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:40.865 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:40.865+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:40.935 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:40.934+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:41.845 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:41.845+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:41.919 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:41.919+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:42.881 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:42.881+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:42.882 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:42.881+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:43.918 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:43.917+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:43.926 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:43.926+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:44.910 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:44.909+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:44.953 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:44.952+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:45.927 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:45.926+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:45.957 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:45.956+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:46.941 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:46.940+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:47.007 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:47.007+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:47.971 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:47.970+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:48.009 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:48.009+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:48.986 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:48.985+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:49.013 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:49.011+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:49.938 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:49.936+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:50.019 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:50.017+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:50.941 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:50.940+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:51.018 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:51.016+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:51.933 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:51.932+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:52.024 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:52.023+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:52.954 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:52.953+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:53.070 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:53.068+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:53.911 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:53.910+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:54.101 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:54.100+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:54.906 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:54.904+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:55.134 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:55.133+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:55.883 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:55.882+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:56.101 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:56.100+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:56.925 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:56.924+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:57.098 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:57.097+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:57.929 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:57.928+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:58.097 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:58.096+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:58.899 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:58.897+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:59.056 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:06:59.054+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:06:59.941 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:06:59.940+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:00.093 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:00.092+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:00.959 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:00.958+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:01.105 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:01.103+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:01.959 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:01.958+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:02.119 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:02.118+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:02.995 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:02.993+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:03.147 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:03.146+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:03.955 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:03.953+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:04.196 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:04.195+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:04.970 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:04.968+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:05.198 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:05.197+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:05.927 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:05.925+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:06.164 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:06.163+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:06.910 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:06.909+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:07.194 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:07.193+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:07.878 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:07.877+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:08.168 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:08.167+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:08.882 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:08.881+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:09.155 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:09.154+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:09.849 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:09.847+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:10.183 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:10.181+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:10.869 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:10.867+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:11.215 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:11.214+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:11.832 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:11.831+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:12.247 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:12.246+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:12.862 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:12.859+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:13.257 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:13.256+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:13.908 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:13.907+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:14.226 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:14.224+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:14.868 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:14.866+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:15.264 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:15.263+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:15.900 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:15.898+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:16.294 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:16.293+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:16.918 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:16.916+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:17.283 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:17.282+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:17.871 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:17.870+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:18.308 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:18.307+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:18.922 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:18.921+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:19.340 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:19.339+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:19.968 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:19.966+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:20.354 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:20.353+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:21.008 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:21.008+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:21.350 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:21.350+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:22.029 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:22.028+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:22.342 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:22.342+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:23.015 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:23.015+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:23.324 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:23.324+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:24.050 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:24.050+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:24.294 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:24.294+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:25.043 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:25.043+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:25.286 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:25.286+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:26.059 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:26.059+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:26.299 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:26.299+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:27.095 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:27.094+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:27.334 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:27.334+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:28.069 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:28.068+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:28.304 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:28.304+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:29.060 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:29.059+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:29.318 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:29.318+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:30.030 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:30.030+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:30.320 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:30.319+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:31.028 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:31.028+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:31.270 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:31.270+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:32.076 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:32.075+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:32.236 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:32.236+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:33.085 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:33.085+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:33.237 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:33.236+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:34.084 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:34.083+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:34.202 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:34.201+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:35.054 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:35.054+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:35.158 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:35.158+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:36.030 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:36.030+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:36.196 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:36.196+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:37.053 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:37.052+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:37.153 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:37.152+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:38.043 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:38.042+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:38.144 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:38.143+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:39.088 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:39.087+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:39.145 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:39.144+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:40.089 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:40.088+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:40.119 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:40.117+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:41.060 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:41.059+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:41.096 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:41.095+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:42.081 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:42.079+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:42.130 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:42.128+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:43.082 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:43.080+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:43.100 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:43.098+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:44.115 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:44.114+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:44.120 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:44.119+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:45.102 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:45.100+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:45.124 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:45.123+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:46.054 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:46.053+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:46.165 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:46.163+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:47.086 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:47.085+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:47.187 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:47.185+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:48.094 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:48.093+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:48.208 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:48.206+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:49.102 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:49.100+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:49.254 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:49.253+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:50.096 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:50.095+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:50.293 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:50.291+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:51.126 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:51.125+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:51.291 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:51.290+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:52.153 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:52.151+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:52.252 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:52.250+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:53.147 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:53.146+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:53.246 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:53.244+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:54.106 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:54.105+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:54.286 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:54.284+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:55.138 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:55.136+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:55.314 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:55.312+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:56.166 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:56.164+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:56.270 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:56.269+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:57.149 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:57.148+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:57.253 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:57.252+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:58.194 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:58.193+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:58.296 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:58.294+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:59.207 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:07:59.205+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:07:59.262 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:07:59.261+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:00.196 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:00.195+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:00.215 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:00.214+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:01.234 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:01.229+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:01.262 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:01.261+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:02.208 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:02.207+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:02.229 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:02.226+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:03.225 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:03.224+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:03.229 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:03.227+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:04.200 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:04.198+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:04.220 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:04.219+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:05.178 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:05.177+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:05.204 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:05.203+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:06.163 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:06.162+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:06.181 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:06.180+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:07.211 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:07.210+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:07.219 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:07.218+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:08.180 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:08.179+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:08.198 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:08.197+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:09.145 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:09.143+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:09.182 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:09.181+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:10.157 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:10.156+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:10.181 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:10.179+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:11.173 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:11.172+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:11.176 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:11.176+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:12.203 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:12.203+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:12.210 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:12.210+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:13.173 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:13.173+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:13.227 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:13.226+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:14.143 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:14.143+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:14.184 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:14.184+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:15.135 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:15.135+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:15.232 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:15.232+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:16.117 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:16.117+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:16.226 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:16.225+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:17.138 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:17.138+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:17.218 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:17.218+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:18.186 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:18.186+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:18.240 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:18.240+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:19.236 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:19.235+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:19.259 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:19.259+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:20.240 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:20.240+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:20.246 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:20.246+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:21.237 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:21.236+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:21.280 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:21.279+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:22.190 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:22.190+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:22.250 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:22.250+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:23.176 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:23.176+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:23.213 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:23.213+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:24.194 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:24.194+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:24.222 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:24.222+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:25.211 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:25.211+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:25.235 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:25.235+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:26.209 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:26.209+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:26.223 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:26.223+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:27.193 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:27.193+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:27.229 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:27.229+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:28.203 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:28.202+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:28.205 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:28.205+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:29.159 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:29.157+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:29.212 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:29.211+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:30.116 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:30.115+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:30.172 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:30.170+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:31.155 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:31.154+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:31.156 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:31.154+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:32.168 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:32.166+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:32.204 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:32.202+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:33.144 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:33.143+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:33.199 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:33.198+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:34.175 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:34.173+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:34.189 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:34.187+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:35.163 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:35.161+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:35.178 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:35.177+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:36.171 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:36.170+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:36.196 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:36.195+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:37.178 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:37.177+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:37.202 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:37.201+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:38.198 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:38.196+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:38.248 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:38.247+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:39.175 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:39.173+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:39.256 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:39.255+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:40.198 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:40.197+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:40.260 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:40.258+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:41.178 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:41.174+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:41.234 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:41.232+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:42.146 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:42.145+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:42.250 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:42.248+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:43.166 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:43.164+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:43.245 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:43.244+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:44.170 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:44.168+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:44.268 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:44.267+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:45.199 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:45.198+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:45.254 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:45.253+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:46.184 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:46.183+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:46.283 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:46.281+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:47.152 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:47.150+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:47.326 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:47.324+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:48.178 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:48.176+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:48.376 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:48.374+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:49.210 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:49.208+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:49.387 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:49.386+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:50.241 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:50.240+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:50.367 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:50.365+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:51.247 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:51.245+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:51.320 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:51.319+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:52.293 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:52.291+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:52.360 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:52.358+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:53.313 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:53.312+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:53.335 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:53.334+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:54.317 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:54.315+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:54.379 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:54.378+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:55.294 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:55.292+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:55.389 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:55.388+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:56.314 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:56.312+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:56.382 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:56.381+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:57.333 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:57.332+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:57.354 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:57.353+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:58.322 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:58.320+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:58.348 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:58.347+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:59.283 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:08:59.282+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:08:59.356 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:08:59.355+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:00.331 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:00.330+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:00.366 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:00.365+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:01.340 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:01.338+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:01.342 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:01.341+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:02.342 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:02.341+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:02.359 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:02.358+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:03.324 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:03.323+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:03.362 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:03.362+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:04.325 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:04.325+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:04.345 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:04.345+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:05.309 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:05.308+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:05.341 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:05.341+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:06.308 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:06.308+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:06.311 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:06.311+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:07.278 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:07.277+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:07.286 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:07.286+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:08.248 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:08.247+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:08.325 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:08.325+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:09.291 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:09.290+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:09.341 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:09.340+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:10.309 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:10.309+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:10.333 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:10.333+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:11.266 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:11.266+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:11.306 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:11.306+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:12.268 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:12.268+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:12.276 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:12.276+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:13.220 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:13.220+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:13.302 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:13.302+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:14.261 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:14.260+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:14.316 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:14.316+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:15.230 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:15.230+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:15.330 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:15.330+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:16.202 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:16.201+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:16.282 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:16.282+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:17.218 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:17.217+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:17.274 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:17.274+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:18.226 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:18.226+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:18.264 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:18.264+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:19.205 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:19.204+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:19.262 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:19.260+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:20.254 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:20.253+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:20.258 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:20.257+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:21.246 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:21.245+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:21.252 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:21.251+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:22.274 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:22.272+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:22.288 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:22.287+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:23.255 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:23.254+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:23.306 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:23.305+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:24.270 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:24.269+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:24.271 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:24.269+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:25.258 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:25.257+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:25.284 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:25.283+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:26.238 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:26.236+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:26.292 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:26.291+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:27.255 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:27.254+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:27.326 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:27.324+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:28.237 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:28.236+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:28.312 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:28.310+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:29.254 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:29.253+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:29.272 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:29.271+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:30.252 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:30.251+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:30.272 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:30.271+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:31.245 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:31.243+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:31.252 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:31.251+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:32.221 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:32.220+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:32.303 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:32.301+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:33.202 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:33.201+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:33.263 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:33.262+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:34.181 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:34.180+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:34.230 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:34.229+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:35.164 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:35.163+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:35.206 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:35.205+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:36.118 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:36.117+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:36.252 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:36.251+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:37.163 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:37.162+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:37.224 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:37.222+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:38.198 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:38.197+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:38.256 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:38.255+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:39.151 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:39.149+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:39.244 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:39.242+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:40.104 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:40.103+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:40.213 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:40.212+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:41.150 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:41.148+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:41.245 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:41.243+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:42.134 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:42.133+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:42.270 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:42.269+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:43.150 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:43.148+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:43.294 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:43.293+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:44.117 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:44.116+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:44.269 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:44.267+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:45.116 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:45.114+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:45.304 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:45.303+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:46.164 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:46.163+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:46.300 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:46.299+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:47.144 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:47.143+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:47.258 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:47.257+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:48.148 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:48.146+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:48.263 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:48.262+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:49.144 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:49.143+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:49.258 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:49.257+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:50.168 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:50.167+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:50.290 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:50.289+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:51.198 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:51.197+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:51.262 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:51.261+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:52.157 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:52.157+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:52.239 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:52.239+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:53.159 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:53.158+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:53.211 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:53.211+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:54.185 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:54.185+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:54.209 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:54.209+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:55.164 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:55.164+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:55.211 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:55.211+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:56.148 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:56.147+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:56.217 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:56.217+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:57.182 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:57.181+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:57.190 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:57.190+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:58.146 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:58.146+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:58.206 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:58.206+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:59.143 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:09:59.143+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:09:59.250 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:09:59.250+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:00.125 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:00.124+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:00.256 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:00.256+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:01.144 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:01.144+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:01.237 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:01.236+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:02.133 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:02.133+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:02.231 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:02.231+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:03.183 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:03.183+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:03.231 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:03.231+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:04.141 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:04.141+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:04.187 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:04.187+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:05.143 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:05.143+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:05.172 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:05.172+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:06.160 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:06.160+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:06.220 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:06.219+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:07.170 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:07.170+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:07.246 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:07.246+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:08.211 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:08.211+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:08.238 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:08.238+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:09.246 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:09.245+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:09.270 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:09.269+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:10.202 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:10.200+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:10.235 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:10.234+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:11.207 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:11.206+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:11.223 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:11.222+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:12.188 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:12.186+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:12.270 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:12.269+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:13.224 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:13.222+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:13.236 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:13.235+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:14.238 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:14.237+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:14.274 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:14.273+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:15.230 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:15.228+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:15.279 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:15.277+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:16.204 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:16.203+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:16.311 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:16.310+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:17.178 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:17.176+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:17.308 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:17.306+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:18.174 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:18.172+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:18.337 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:18.336+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:19.151 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:19.150+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:19.341 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:19.340+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:20.110 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:20.109+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:20.334 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:20.333+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:21.120 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:21.119+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:21.340 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:21.339+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:22.119 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:22.118+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:22.319 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:22.317+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:23.137 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:23.136+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:23.339 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:23.338+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:24.119 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:24.118+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:24.362 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:24.361+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:25.145 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:25.144+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:25.384 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:25.383+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:26.175 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:26.174+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:26.431 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:26.430+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:27.179 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:27.177+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:27.435 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:27.434+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:28.201 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:28.200+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:28.457 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:28.456+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:29.209 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:29.207+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:29.499 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:29.498+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:30.195 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:30.193+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:30.481 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:30.480+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:31.167 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:31.166+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:31.483 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:31.482+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:32.211 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:32.210+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:32.483 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:32.481+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:33.253 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:33.252+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:33.517 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:33.516+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:34.253 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:34.252+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:34.543 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:34.542+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:35.217 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:35.216+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:35.517 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:35.516+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:36.180 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:36.178+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:36.566 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:36.564+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:37.171 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:37.170+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:37.559 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:37.558+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:38.157 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:38.155+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:38.549 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:38.548+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:39.188 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:39.187+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:39.502 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:39.501+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:40.223 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:40.222+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:40.492 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:40.490+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:41.186 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:41.185+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:41.483 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:41.482+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:42.226 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:42.226+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:42.532 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:42.531+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:43.274 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:43.273+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:43.548 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:43.547+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:44.232 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:44.232+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:44.576 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:44.576+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:45.217 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:45.217+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:45.560 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:45.560+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:46.248 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:46.248+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:46.553 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:46.553+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:47.215 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:47.215+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:47.532 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:47.532+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:48.264 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:48.264+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:48.580 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:48.580+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:49.278 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:49.277+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:49.588 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:49.588+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:50.243 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:50.243+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:50.545 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:50.545+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:51.232 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:51.232+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:51.575 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:51.575+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:52.211 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:52.211+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:52.541 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:52.541+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:53.165 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:53.165+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:53.564 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:53.564+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:54.130 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:54.130+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:54.587 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:54.587+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:55.160 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:55.160+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:55.638 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:55.638+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:56.193 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:56.193+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:56.634 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:56.634+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:57.153 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:57.152+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:57.663 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:57.662+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:58.111 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:58.110+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:58.642 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:58.642+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:59.106 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:10:59.106+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:10:59.639 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:10:59.638+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:00.139 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:00.137+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:00.689 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:00.687+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:01.091 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:01.090+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:01.715 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:01.714+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:02.056 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:02.055+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:02.707 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:02.705+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:03.068 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:03.066+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:03.746 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:03.744+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:04.052 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:04.051+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:04.774 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:04.773+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:05.038 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:05.037+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:05.822 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:05.821+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:06.028 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:06.027+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:06.861 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:06.859+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:07.020 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:07.019+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:07.889 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:07.888+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:08.023 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:08.021+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:08.919 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:08.918+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:08.992 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:08.991+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:09.939 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:09.938+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:10.025 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:10.023+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:10.891 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:10.890+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:11.047 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:11.045+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:11.942 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:11.940+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:12.044 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:12.043+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:12.931 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:12.930+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:13.068 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:13.067+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:13.959 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:13.957+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:14.070 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:14.069+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:14.964 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:14.963+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:15.030 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:15.029+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:15.933 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:15.932+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:16.025 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:16.024+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:16.966 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:16.965+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:16.993 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:16.992+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:17.953 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:17.951+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:17.972 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:17.971+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:18.914 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:18.913+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:18.968 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:18.967+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:19.918 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:19.917+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:19.985 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:19.983+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:20.913 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:20.912+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:21.007 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:21.006+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:21.903 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:21.901+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:21.989 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:21.987+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:22.890 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:22.889+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:23.037 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:23.036+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:23.861 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:23.859+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:24.043 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:24.042+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:24.868 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:24.867+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:25.040 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:25.038+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:25.878 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:25.877+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:25.992 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:25.991+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:26.869 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:26.868+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:27.034 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:27.033+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:27.915 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:27.913+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:28.009 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:28.008+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:28.942 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:28.941+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:29.001 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:28.999+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:29.942 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:29.941+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:30.019 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:30.018+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:30.947 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:30.946+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:31.023 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:31.022+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:31.988 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:31.987+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:32.063 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:32.062+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:32.990 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:32.989+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:33.091 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:33.091+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:33.963 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:33.962+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:34.051 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:34.051+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:34.928 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:34.928+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:35.098 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:35.098+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:35.958 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:35.957+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:36.054 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:36.053+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:36.923 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:36.922+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:37.090 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:37.089+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:37.893 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:37.893+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:38.074 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:38.074+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:38.894 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:38.894+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:39.051 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:39.050+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:39.890 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:39.890+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:40.025 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:40.025+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:40.934 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:40.934+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:41.000 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:41.000+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:41.958 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:41.958+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:42.047 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:42.046+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:42.955 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:42.955+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:42.998 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:42.998+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:43.981 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:43.981+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:44.038 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:44.038+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:45.031 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:45.031+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:45.062 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:45.062+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:46.009 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:46.009+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:46.074 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:46.073+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:46.994 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:46.993+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:47.070 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:47.070+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:47.997 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:47.996+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:48.068 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:48.068+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:48.994 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:48.994+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:49.050 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:49.049+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:50.010 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:50.009+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:50.052 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:50.051+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:50.982 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:50.980+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:51.022 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:51.021+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:52.006 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:52.004+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:52.030 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:52.029+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:52.977 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:52.976+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:53.060 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:53.059+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:54.010 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:54.008+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:54.062 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:54.061+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:55.034 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:55.032+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:55.047 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:55.046+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:56.000 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:55.999+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:56.058 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:56.057+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:56.978 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:56.976+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:57.019 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:57.018+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:57.992 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:57.990+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:58.001 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:57.999+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:58.964 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:58.963+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:59.034 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:11:59.033+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:11:59.978 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:11:59.977+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:00.002 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:00.001+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:00.947 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:00.946+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:01.048 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:01.047+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:01.950 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:01.948+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:02.096 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:02.095+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:02.994 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:02.992+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:03.087 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:03.086+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:03.956 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:03.954+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:04.103 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:04.102+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:04.996 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:04.995+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:05.122 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:05.121+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:06.034 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:06.033+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:06.148 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:06.147+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:07.081 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:07.079+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:07.107 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:07.105+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:08.038 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:08.037+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:08.125 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:08.124+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:09.022 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:09.021+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:09.157 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:09.155+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:10.013 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:10.011+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:10.162 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:10.160+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:10.988 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:10.987+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:11.162 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:11.161+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:12.026 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:12.024+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:12.120 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:12.119+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:13.069 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:13.067+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:13.122 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:13.120+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:14.064 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:14.063+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:14.074 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:14.073+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:15.027 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:15.025+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:15.078 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:15.076+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:16.052 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:16.051+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:16.061 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:16.060+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:17.048 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:17.046+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:17.058 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:17.057+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:18.030 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:18.029+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:18.062 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:18.060+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:19.077 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:19.075+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:19.097 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:19.095+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:20.113 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:20.111+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:20.134 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:20.133+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:21.107 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:21.106+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:21.157 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:21.156+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:22.101 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:22.100+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:22.131 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:22.130+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:23.113 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:23.112+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:23.130 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:23.130+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:24.130 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:24.130+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:24.180 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:24.180+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:25.147 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:25.147+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:25.168 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:25.168+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:26.153 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:26.153+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:26.162 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:26.161+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:27.201 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:27.201+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:27.211 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:27.211+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:28.211 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:28.211+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:28.228 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:28.228+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:29.205 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:29.205+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:29.238 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:29.238+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:30.160 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:30.160+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:30.267 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:30.267+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:31.152 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:31.151+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:31.273 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:31.272+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:32.141 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:32.141+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:32.308 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:32.307+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:33.094 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:33.094+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:33.298 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:33.298+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:34.058 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:34.058+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:34.275 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:34.275+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:35.058 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:35.057+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:35.275 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:35.274+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:36.013 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:36.012+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:36.226 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:36.226+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:36.985 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:36.985+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:37.252 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:37.252+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:38.034 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:38.034+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:38.268 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:38.268+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:39.036 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:39.035+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:39.317 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:39.317+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:40.006 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:40.005+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:40.286 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:40.286+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:40.987 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:40.986+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:41.300 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:41.299+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:41.965 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:41.963+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:42.261 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:42.259+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:42.935 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:42.933+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:43.268 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:43.267+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:43.914 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:43.913+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:44.296 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:44.294+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:44.867 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:44.866+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:45.312 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:45.311+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:45.881 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:45.880+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:46.293 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:46.291+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:46.925 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:46.924+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:47.341 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:47.340+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:47.920 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:47.918+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:48.349 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:48.348+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:48.927 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:48.926+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:49.363 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:49.361+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:49.973 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:49.971+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:50.409 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:50.407+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:50.997 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:50.995+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:51.397 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:51.395+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:52.023 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:52.022+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:52.444 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:52.443+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:53.056 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:53.054+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:53.431 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:53.429+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:54.085 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:54.084+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:54.442 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:54.441+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:55.050 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:55.049+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:55.446 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:55.444+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:56.090 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:56.089+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:56.449 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:56.448+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:57.111 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:57.109+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:57.483 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:57.482+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:58.096 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:58.095+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:58.453 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:58.452+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:59.114 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:12:59.113+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:12:59.479 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:12:59.477+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:00.160 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:00.159+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:00.458 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:00.457+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:01.153 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:01.152+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:01.456 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:01.455+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:02.108 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:02.107+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:02.491 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:02.489+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:03.096 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:03.095+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:03.457 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:03.455+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:04.053 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:04.051+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:04.410 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:04.409+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:05.102 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:05.101+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:05.362 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:05.360+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:06.120 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:06.119+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:06.375 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:06.374+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:07.141 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:07.140+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:07.404 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:07.403+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:08.142 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:08.141+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:08.422 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:08.421+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:09.146 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:09.145+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:09.466 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:09.464+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:10.126 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:10.125+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:10.471 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:10.470+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:11.128 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:11.126+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:11.464 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:11.463+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:12.089 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:12.088+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:12.433 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:12.432+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:13.074 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:13.074+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:13.437 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:13.437+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:14.037 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:14.037+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:14.420 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:14.419+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:15.011 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:15.011+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:15.423 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:15.422+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:16.049 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:16.048+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:16.421 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:16.421+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:17.082 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:17.081+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:17.395 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:17.394+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:18.114 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:18.114+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:18.369 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:18.369+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:19.142 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:19.142+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:19.401 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:19.400+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:20.173 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:20.173+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:20.385 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:20.385+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:21.209 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:21.208+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:21.350 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:21.349+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:22.210 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:22.209+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:22.341 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:22.341+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:23.195 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:23.195+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:23.377 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:23.376+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:24.241 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:24.241+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:24.358 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:24.358+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:25.271 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:25.271+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:25.392 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:25.392+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:26.231 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:26.231+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:26.356 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:26.355+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:27.209 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:27.209+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:27.344 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:27.344+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:28.209 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:28.209+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:28.309 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:28.308+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:29.185 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:29.185+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:29.285 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:29.285+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:30.179 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:30.179+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:30.240 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:30.240+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:31.188 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:31.186+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:31.266 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:31.265+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:32.215 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:32.213+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:32.239 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:32.238+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:33.179 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:33.177+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:33.284 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:33.282+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:34.134 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:34.133+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:34.269 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:34.267+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:35.112 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:35.111+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:35.249 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:35.248+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:36.080 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:36.079+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:36.269 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:36.267+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:37.063 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:37.061+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:37.261 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:37.260+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:38.056 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:38.055+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:38.277 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:38.275+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:39.064 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:39.063+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:39.271 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:39.269+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:40.027 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:40.026+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:40.307 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:40.306+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:41.059 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:41.058+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:41.276 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:41.275+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:42.052 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:42.050+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:42.251 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:42.249+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:43.083 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:43.081+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:43.229 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:43.228+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:44.051 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:44.050+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:44.263 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:44.262+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:45.068 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:45.066+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:45.229 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:45.227+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:46.061 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:46.060+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:46.267 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:46.266+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:47.076 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:47.075+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:47.244 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:47.242+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:48.113 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:48.112+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:48.269 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:48.268+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:49.106 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:49.104+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:49.281 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:49.280+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:50.076 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:50.075+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:50.263 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:50.262+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:51.033 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:51.031+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:51.269 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:51.268+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:52.046 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:52.045+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:52.261 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:52.259+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:53.005 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:53.003+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:53.230 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:53.228+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:54.031 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:54.030+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:54.260 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:54.259+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:54.994 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:54.992+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:55.241 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:55.240+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:55.991 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:55.990+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:56.199 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:56.197+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:56.995 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:56.994+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:57.249 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:57.247+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:57.951 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:57.950+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:58.215 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:58.213+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:58.974 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:58.973+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:59.264 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:13:59.263+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:13:59.980 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:13:59.979+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:00.265 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:00.264+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:00.941 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:00.939+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:01.262 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:01.258+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:01.932 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:01.931+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:02.239 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:02.237+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:02.962 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:02.962+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:03.278 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:03.277+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:03.959 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:03.958+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:04.309 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:04.309+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:05.001 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:05.000+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:05.333 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:05.333+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:05.979 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:05.979+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:06.369 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:06.369+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:06.951 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:06.950+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:07.352 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:07.351+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:07.990 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:07.989+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:08.319 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:08.319+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:08.988 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:08.988+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:09.271 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:09.271+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:09.987 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:09.987+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:10.307 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:10.307+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:10.981 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:10.980+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:11.261 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:11.261+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:12.016 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:12.015+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:12.264 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:12.263+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:13.003 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:13.002+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:13.297 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:13.297+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:13.974 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:13.973+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:14.303 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:14.303+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:15.013 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:15.013+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:15.351 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:15.350+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:16.008 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:16.008+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:16.312 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:16.312+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:16.987 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:16.987+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:17.361 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:17.361+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:18.030 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:18.030+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:18.411 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:18.410+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:19.079 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:19.078+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:19.440 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:19.439+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:20.044 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:20.043+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:20.429 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:20.428+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:21.040 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:21.039+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:21.468 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:21.466+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:22.029 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:22.028+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:22.458 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:22.457+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:22.987 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:22.985+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:23.448 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:23.447+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:23.958 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:23.956+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:24.418 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:24.417+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:24.991 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:24.990+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:25.383 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:25.382+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:25.997 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:25.995+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:26.418 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:26.417+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:26.995 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:26.994+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:27.393 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:27.391+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:28.020 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:28.018+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:28.428 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:28.426+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:28.997 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:28.996+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:29.401 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:29.400+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:29.983 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:29.982+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:30.391 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:30.389+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:31.001 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:31.000+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:31.405 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:31.404+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:32.001 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:31.999+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:32.451 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:32.450+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:33.022 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:33.020+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:33.499 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:33.498+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:34.009 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:34.008+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:34.518 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:34.517+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:35.051 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:35.049+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:35.528 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:35.527+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:36.081 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:36.079+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:36.527 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:36.526+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:37.049 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:37.048+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:37.535 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:37.534+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:38.063 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:38.062+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:38.570 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:38.569+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:39.112 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:39.110+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:39.582 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:39.578+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:40.130 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:40.128+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:40.531 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:40.530+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:41.127 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:41.125+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:41.563 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:41.562+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:42.119 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:42.117+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:42.567 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:42.565+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:43.169 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:43.167+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:43.534 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:43.533+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:44.209 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:44.208+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:44.573 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:44.572+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:45.226 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:45.224+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:45.601 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:45.599+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:46.263 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:46.262+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:46.601 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:46.600+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:47.277 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:47.276+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:47.621 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:47.620+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:48.235 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:48.233+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:48.596 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:48.594+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:49.229 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:49.227+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:49.572 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:49.571+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:50.211 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:50.209+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:50.608 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:50.607+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:51.235 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:51.234+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:51.568 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:51.566+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:52.248 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:52.247+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:52.597 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:52.596+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:53.216 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:53.215+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:53.638 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:53.637+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:54.257 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:54.256+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:54.595 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:54.595+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:55.273 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:55.273+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:55.638 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:55.637+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:56.297 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:56.297+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:56.677 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:56.676+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:57.310 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:57.310+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:57.688 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:57.687+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:58.313 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:58.312+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:58.656 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:58.655+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:59.287 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:14:59.287+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:14:59.684 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:14:59.683+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:00.332 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:00.331+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:00.681 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:00.681+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:01.378 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:01.378+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:01.726 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:01.726+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:02.337 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:02.336+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:02.758 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:02.758+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:03.360 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:03.359+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:03.734 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:03.734+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:04.358 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:04.357+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:04.778 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:04.778+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:05.340 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:05.340+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:05.799 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:05.799+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:06.301 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:06.301+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:06.833 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:06.831+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:07.343 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:07.343+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:07.786 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:07.786+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:08.328 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:08.327+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:08.769 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:08.768+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:09.351 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:09.351+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:09.739 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:09.738+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:10.356 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:10.356+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:10.690 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:10.690+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:11.401 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:11.400+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:11.664 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:11.663+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:12.408 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:12.407+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:12.619 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:12.618+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:13.435 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:13.433+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:13.631 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:13.629+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:14.471 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:14.470+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:14.678 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:14.677+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:15.449 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:15.447+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:15.724 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:15.723+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:16.445 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:16.444+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:16.707 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:16.706+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:17.473 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:17.472+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:17.681 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:17.680+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:18.437 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:18.436+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:18.662 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:18.661+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:19.469 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:19.467+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:19.633 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:19.632+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:20.441 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:20.440+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:20.677 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:20.675+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:21.423 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:21.421+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:21.723 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:21.722+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:22.408 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:22.406+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:22.696 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:22.695+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:23.382 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:23.380+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:23.679 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:23.677+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:24.415 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:24.414+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:24.686 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:24.685+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:25.377 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:25.376+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:25.642 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:25.641+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:26.396 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:26.394+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:26.674 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:26.672+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:27.374 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:27.373+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:27.681 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:27.679+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:28.370 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:28.369+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:28.674 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:28.673+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:29.349 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:29.348+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:29.717 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:29.716+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:30.341 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:30.339+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:30.730 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:30.729+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:31.295 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:31.294+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:31.756 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:31.755+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:32.338 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:32.336+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:32.729 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:32.727+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:33.289 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:33.288+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:33.739 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:33.738+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:34.257 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:34.256+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:34.790 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:34.789+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:35.247 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:35.246+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:35.755 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:35.754+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:36.206 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:36.205+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:36.721 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:36.719+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:37.205 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:37.203+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:37.697 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:37.696+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:38.221 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:38.220+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:38.690 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:38.689+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:39.266 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:39.265+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:39.698 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:39.697+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:40.294 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:40.292+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:40.740 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:40.738+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:41.260 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:41.259+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:41.756 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:41.754+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:42.215 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:42.213+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:42.717 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:42.716+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:43.263 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:43.262+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:43.711 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:43.709+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:44.218 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:44.217+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:44.661 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:44.661+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:45.214 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:45.214+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:45.659 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:45.659+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:46.221 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:46.221+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:46.615 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:46.615+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:47.240 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:47.240+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:47.569 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:47.569+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:48.235 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:48.235+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:48.537 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:48.536+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:49.260 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:49.260+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:49.534 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:49.534+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:50.276 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:50.276+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:50.581 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:50.581+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:51.305 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:51.305+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:51.575 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:51.575+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:52.284 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:52.283+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:52.543 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:52.543+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:53.275 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:53.274+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:53.546 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:53.545+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:54.289 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:54.289+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:54.539 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:54.539+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:55.302 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:55.301+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:55.497 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:55.497+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:56.338 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:56.337+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:56.474 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:56.474+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:57.388 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:57.388+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:57.441 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:57.441+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:58.396 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:58.395+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:58.438 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:58.438+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:59.377 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:15:59.376+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:15:59.483 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:15:59.482+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:00.358 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:00.357+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:00.488 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:00.487+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:01.360 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:01.359+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:01.474 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:01.474+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:02.379 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:02.378+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:02.460 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:02.459+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:03.411 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:03.410+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:03.449 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:03.447+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:04.432 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:04.431+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:04.477 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:04.475+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:05.427 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:05.426+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:05.451 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:05.450+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:06.415 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:06.414+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:06.444 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:06.443+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:07.374 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:07.373+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:07.442 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:07.440+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:08.334 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:08.333+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:08.483 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:08.482+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:09.384 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:09.382+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:09.461 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:09.460+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:10.404 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:10.403+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:10.466 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:10.465+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:11.423 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:11.422+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:11.504 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:11.503+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:12.472 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:12.471+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:12.473 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:12.471+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:13.454 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:13.453+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:13.465 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:13.464+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:14.463 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:14.461+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:14.464 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:14.462+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:15.457 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:15.456+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:15.485 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:15.484+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:16.500 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:16.499+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:16.503 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:16.502+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:17.501 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:17.500+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:17.548 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:17.547+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:18.495 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:18.493+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:18.550 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:18.548+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:19.475 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:19.474+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:19.510 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:19.509+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:20.459 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:20.458+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:20.520 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:20.518+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:21.438 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:21.436+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:21.508 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:21.506+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:22.460 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:22.458+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:22.482 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:22.481+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:23.474 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:23.473+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:23.482 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:23.481+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:24.504 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:24.502+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:24.510 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:24.509+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:25.523 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:25.522+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:25.551 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:25.550+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:26.500 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:26.499+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:26.512 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:26.511+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:27.481 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:27.480+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:27.536 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:27.535+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:28.452 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:28.450+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:28.561 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:28.560+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:29.483 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:29.482+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:29.512 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:29.511+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:30.529 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:30.528+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:30.559 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:30.558+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:31.538 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:31.536+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:31.548 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:31.547+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:32.540 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:32.539+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:32.573 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:32.572+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:33.504 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:33.503+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:33.583 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:33.582+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:34.494 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:34.494+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:34.570 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:34.569+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:35.530 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:35.529+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:35.610 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:35.609+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:36.505 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:36.504+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:36.601 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:36.601+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:37.491 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:37.491+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:37.599 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:37.599+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:38.451 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:38.450+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:38.577 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:38.577+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:39.442 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:39.442+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:39.605 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:39.604+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:40.401 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:40.401+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:40.600 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:40.600+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:41.406 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:41.406+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:41.625 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:41.624+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:42.414 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:42.413+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:42.667 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:42.667+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:43.386 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:43.386+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:43.663 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:43.663+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:44.433 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:44.432+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:44.678 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:44.678+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:45.444 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:45.444+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:45.648 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:45.647+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:46.417 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:46.417+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:46.697 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:46.697+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:47.451 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:47.450+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:47.698 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:47.698+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:48.500 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:48.499+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:48.668 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:48.668+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:49.511 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:49.510+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:49.622 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:49.622+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:50.530 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:50.530+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:50.600 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:50.599+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:51.560 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:51.560+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:51.572 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:51.571+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:52.536 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:52.535+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:52.613 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:52.612+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:53.564 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:53.563+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:53.576 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:53.574+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:54.526 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:54.525+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:54.581 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:54.579+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:55.556 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:55.555+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:55.567 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:55.566+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:56.534 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:56.533+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:56.536 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:56.535+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:57.516 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:57.515+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:57.518 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:57.517+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:58.515 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:58.513+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:58.532 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:58.530+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:59.489 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:16:59.487+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:16:59.528 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:16:59.526+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:00.506 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:00.505+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:00.520 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:00.519+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:01.517 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:01.515+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:01.537 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:01.536+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:02.533 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:02.531+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:02.584 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:02.582+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:03.512 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:03.511+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:03.551 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:03.549+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:04.508 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:04.507+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:04.528 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:04.527+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:05.514 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:05.513+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:05.524 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:05.523+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:06.489 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:06.487+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:06.563 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:06.561+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:07.475 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:07.474+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:07.594 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:07.592+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:08.464 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:08.463+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:08.582 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:08.581+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:09.514 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:09.513+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:09.542 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:09.541+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:10.510 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:10.509+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:10.534 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:10.533+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:11.494 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:11.492+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:11.552 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:11.551+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:12.502 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:12.500+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:12.568 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:12.567+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:13.552 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:13.551+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:13.603 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:13.602+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:14.512 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:14.511+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:14.606 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:14.604+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:15.543 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:15.542+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:15.566 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:15.565+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:16.573 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:16.572+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:16.587 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:16.586+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:17.540 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:17.539+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:17.586 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:17.585+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:18.545 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:18.544+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:18.568 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:18.567+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:19.541 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:19.540+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:19.558 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:19.556+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:20.518 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:20.517+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:20.518 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:20.517+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:21.483 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:21.481+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:21.524 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:21.523+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:22.504 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:22.502+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:22.519 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:22.518+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:23.499 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:23.499+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:23.504 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:23.503+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:24.534 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:24.533+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:24.543 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:24.543+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:25.495 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:25.495+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:25.520 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:25.520+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:26.455 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:26.454+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:26.517 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:26.500+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:27.457 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:27.457+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:27.490 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:27.490+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:28.411 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:28.411+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:28.540 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:28.539+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:29.383 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:29.382+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:29.495 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:29.494+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:30.367 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:30.367+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:30.515 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:30.515+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:31.362 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:31.361+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:31.543 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:31.543+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:32.333 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:32.333+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:32.575 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:32.575+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:33.325 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:33.325+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:33.581 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:33.581+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:34.320 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:34.320+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:34.553 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:34.553+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:35.367 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:35.367+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:35.521 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:35.521+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:36.354 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:36.353+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:36.537 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:36.536+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:37.335 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:37.335+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:37.568 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:37.568+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:38.322 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:38.321+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:38.545 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:38.544+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:39.327 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:39.327+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:39.541 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:39.541+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:40.353 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:40.352+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:40.523 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:40.522+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:41.333 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:41.333+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:41.548 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:41.548+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:42.359 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:42.357+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:42.582 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:42.580+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:43.337 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:43.335+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:43.568 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:43.567+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:44.297 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:44.296+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:44.550 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:44.549+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:45.323 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:45.322+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:45.532 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:45.531+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:46.365 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:46.364+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:46.551 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:46.549+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:47.369 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:47.368+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:47.528 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:47.527+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:48.348 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:48.347+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:48.538 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:48.536+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:49.319 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:49.318+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:49.541 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:49.540+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:50.319 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:50.318+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:50.547 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:50.545+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:51.331 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:51.330+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:51.516 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:51.515+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:52.365 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:52.363+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:52.475 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:52.474+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:53.393 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:53.392+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:53.459 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:53.458+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:54.441 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:54.440+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:54.496 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:54.495+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:55.451 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:55.450+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:55.517 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:55.516+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:56.499 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:56.497+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:56.541 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:56.540+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:57.491 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:57.489+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:57.587 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:57.586+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:58.500 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:58.499+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:58.618 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:58.617+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:59.494 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:17:59.493+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:17:59.576 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:17:59.575+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:00.451 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:00.450+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:00.553 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:00.552+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:01.423 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:01.422+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:01.593 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:01.592+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:02.386 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:02.385+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:02.545 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:02.544+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:03.410 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:03.409+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:03.517 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:03.516+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:04.387 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:04.385+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:04.559 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:04.558+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:05.376 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:05.375+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:05.585 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:05.584+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:06.349 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:06.347+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:06.548 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:06.547+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:07.366 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:07.365+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:07.572 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:07.570+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:08.389 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:08.388+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:08.602 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:08.601+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:09.442 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:09.439+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:09.613 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:09.612+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:10.436 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:10.435+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:10.662 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:10.661+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:11.431 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:11.430+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:11.634 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:11.633+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:12.454 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:12.452+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:12.635 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:12.634+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:13.500 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:13.498+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:13.676 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:13.674+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:14.540 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:14.539+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:14.680 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:14.679+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:15.534 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:15.533+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:15.639 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:15.638+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:16.550 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:16.550+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:16.604 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:16.604+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:17.564 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:17.564+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:17.584 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:17.583+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:18.572 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:18.571+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:18.599 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:18.599+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:19.528 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:19.528+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:19.598 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:19.598+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:20.518 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:20.517+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:20.606 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:20.606+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:21.548 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:21.547+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:21.621 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:21.621+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:22.520 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:22.520+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:22.594 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:22.594+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:23.532 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:23.532+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:23.614 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:23.613+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:24.500 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:24.500+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:24.638 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:24.638+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:25.530 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:25.529+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:25.661 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:25.661+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:26.492 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:26.492+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:26.709 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:26.708+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:27.475 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:27.474+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:27.756 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:27.756+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:28.518 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:28.518+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:28.786 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:28.786+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:29.548 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:29.548+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:29.821 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:29.821+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:30.586 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:30.586+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:30.851 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:30.851+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:31.630 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:31.630+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:31.865 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:31.865+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:32.630 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:32.629+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:32.837 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:32.835+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:33.675 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:33.674+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:33.833 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:33.831+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:34.696 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:34.695+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:34.872 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:34.871+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:35.711 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:35.710+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:35.830 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:35.828+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:36.711 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:36.709+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:36.800 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:36.798+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:37.758 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:37.757+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:37.824 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:37.823+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:38.775 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:38.774+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:38.866 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:38.864+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:39.779 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:39.778+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:39.873 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:39.872+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:40.757 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:40.756+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:40.920 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:40.919+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:41.736 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:41.734+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:41.900 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:41.899+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:42.690 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:42.688+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:42.918 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:42.916+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:43.720 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:43.719+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:43.887 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:43.886+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:44.714 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:44.713+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:44.916 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:44.915+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:45.676 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:45.674+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:45.913 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:45.912+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:46.630 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:46.629+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:46.880 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:46.879+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:47.643 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:47.642+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:47.900 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:47.899+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:48.636 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:48.635+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:48.910 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:48.908+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:49.642 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:49.641+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:49.954 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:49.953+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:50.602 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:50.601+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:50.917 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:50.916+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:51.601 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:51.600+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:51.913 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:51.912+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:52.624 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:52.623+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:52.926 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:52.924+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:53.670 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:53.669+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:53.936 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:53.935+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:54.669 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:54.667+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:54.974 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:54.972+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:55.681 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:55.679+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:56.021 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:56.020+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:56.688 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:56.687+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:56.988 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:56.987+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:57.689 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:57.688+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:57.998 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:57.997+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:58.706 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:58.705+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:58.990 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:58.988+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:59.750 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:18:59.748+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:18:59.985 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:18:59.984+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:00.759 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:00.758+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:00.997 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:00.996+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:01.753 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:01.752+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:02.038 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:02.037+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:02.804 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:02.802+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:03.014 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:03.012+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:03.764 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:03.763+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:03.980 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:03.979+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:04.727 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:04.727+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:04.951 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:04.950+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:05.741 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:05.741+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:05.946 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:05.946+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:06.764 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:06.763+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:06.908 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:06.907+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:07.796 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:07.796+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:07.890 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:07.890+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:08.781 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:08.780+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:08.842 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:08.842+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:09.829 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:09.829+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:09.884 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:09.884+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:10.848 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:10.848+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:10.879 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:10.879+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:11.833 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:11.832+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:11.859 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:11.859+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:12.838 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:12.838+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:12.880 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:12.879+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:13.820 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:13.819+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:13.834 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:13.834+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:14.820 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:14.820+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:14.825 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:14.825+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:15.783 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:15.783+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:15.869 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:15.869+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:16.815 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:16.815+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:16.850 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:16.850+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:17.810 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:17.810+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:17.811 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:17.811+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:18.768 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:18.768+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:18.824 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:18.823+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:19.773 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:19.773+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:19.821 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:19.821+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:20.732 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:20.732+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:20.783 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:20.783+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:21.742 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:21.742+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:21.743 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:21.743+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:22.704 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:22.702+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:22.785 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:22.784+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:23.683 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:23.682+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:23.747 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:23.746+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:24.696 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:24.695+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:24.787 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:24.786+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:25.732 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:25.731+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:25.820 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:25.818+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:26.692 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:26.691+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:26.835 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:26.834+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:27.650 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:27.649+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:27.861 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:27.860+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:28.605 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:28.604+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:28.864 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:28.863+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:29.624 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:29.622+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:29.890 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:29.889+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:30.580 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:30.579+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:30.919 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:30.918+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:31.558 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:31.557+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:31.877 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:31.876+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:32.531 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:32.529+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:32.897 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:32.896+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:33.506 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:33.505+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:33.917 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:33.915+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:34.553 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:34.552+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:34.877 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:34.876+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:35.521 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:35.520+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:35.832 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:35.831+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:36.527 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:36.526+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:36.863 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:36.862+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:37.541 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:37.540+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:37.830 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:37.829+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:38.498 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:38.497+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:38.785 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:38.784+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:39.462 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:39.460+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:39.790 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:39.789+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:40.471 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:40.469+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:40.837 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:40.836+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:41.472 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:41.471+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:41.865 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:41.864+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:42.447 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:42.445+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:42.843 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:42.842+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:43.470 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:43.469+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:43.808 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:43.807+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:44.475 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:44.474+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:44.773 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:44.770+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:45.481 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:45.479+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:45.820 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:45.819+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:46.476 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:46.475+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:46.808 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:46.806+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:47.477 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:47.476+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:47.805 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:47.804+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:48.522 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:48.521+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:48.804 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:48.802+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:49.569 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:49.568+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:49.799 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:49.798+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:50.568 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:50.567+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:50.769 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:50.768+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:51.593 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:51.591+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:51.742 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:51.741+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:52.557 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:52.556+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:52.745 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:52.744+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:53.526 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:53.525+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:53.728 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:53.727+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:54.511 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:54.510+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:54.772 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:54.771+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:55.496 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:55.496+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:55.763 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:55.762+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:56.457 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:56.457+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:56.772 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:56.772+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:57.469 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:57.469+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:57.794 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:57.793+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:58.495 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:58.495+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:58.843 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:58.843+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:59.477 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:19:59.477+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:19:59.795 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:19:59.794+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:00.474 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:00.474+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:00.778 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:00.778+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:01.470 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:01.469+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:01.812 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:01.811+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:02.449 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:02.448+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:02.841 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:02.840+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:03.410 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:03.410+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:03.798 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:03.797+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:04.364 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:04.363+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:04.778 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:04.778+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:05.378 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:05.377+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:05.736 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:05.736+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:06.388 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:06.388+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:06.748 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:06.748+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:07.354 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:07.353+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:07.700 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:07.700+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:08.357 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:08.356+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:08.744 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:08.744+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:09.333 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:09.333+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:09.780 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:09.780+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:10.287 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:10.286+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:10.733 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:10.732+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:11.324 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:11.323+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:11.772 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:11.772+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:12.294 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:12.294+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:12.811 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:12.810+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:13.264 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:13.263+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:13.802 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:13.801+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:14.273 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:14.272+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:14.826 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:14.824+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:15.269 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:15.267+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:15.832 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:15.830+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:16.224 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:16.223+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:16.877 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:16.876+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:17.264 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:17.263+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:17.904 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:17.902+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:18.254 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:18.252+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:18.950 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:18.949+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:19.291 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:19.289+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:19.996 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:19.995+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:20.294 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:20.293+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:20.952 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:20.950+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:21.280 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:21.279+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:21.942 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:21.941+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:22.314 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:22.313+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:22.919 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:22.917+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:23.340 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:23.338+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:23.896 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:23.894+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:24.356 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:24.355+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:24.884 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:24.882+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:25.366 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:25.365+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:25.927 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:25.925+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:26.353 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:26.350+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:26.965 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:26.962+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:27.370 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:27.369+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:28.008 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:28.006+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:28.388 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:28.387+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:29.032 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:29.031+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:29.410 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:29.408+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:29.995 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:29.994+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:30.371 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:30.370+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:31.001 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:31.000+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:31.328 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:31.327+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:32.007 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:32.006+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:32.378 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:32.376+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:33.008 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:33.007+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:33.396 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:33.394+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:33.996 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:33.995+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:34.414 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:34.413+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:34.988 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:34.987+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:35.376 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:35.374+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:35.952 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:35.951+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:36.342 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:36.341+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:36.982 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:36.981+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:37.353 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:37.351+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:37.978 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:37.977+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:38.380 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:38.379+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:39.003 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:39.002+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:39.373 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:39.371+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:39.990 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:39.989+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:40.352 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:40.351+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:41.032 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:41.030+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:41.357 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:41.356+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:42.011 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:42.009+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:42.368 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:42.366+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:43.010 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:43.008+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:43.410 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:43.409+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:43.983 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:43.982+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:44.453 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:44.452+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:44.999 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:44.998+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:45.449 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:45.447+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:45.960 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:45.960+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:46.494 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:46.494+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:46.950 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:46.950+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:47.502 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:47.502+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:47.928 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:47.928+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:48.470 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:48.470+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:48.928 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:48.928+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:49.428 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:49.428+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:49.958 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:49.957+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:50.387 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:50.387+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:50.981 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:50.980+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:51.349 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:51.349+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:52.011 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:52.011+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:52.393 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:52.393+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:52.965 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:52.965+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:53.435 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:53.435+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:53.994 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:53.994+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:54.416 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:54.416+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:54.986 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:54.985+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:55.437 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:55.437+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:56.032 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:56.032+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:56.440 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:56.439+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:57.007 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:57.007+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:57.436 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:57.435+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:58.034 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:58.034+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:58.454 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:58.454+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:59.073 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:20:59.073+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:20:59.415 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:20:59.415+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:00.095 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:00.095+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:00.408 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:00.408+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:01.130 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:01.130+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:01.427 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:01.426+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:02.152 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:02.152+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:02.440 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:02.439+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:03.120 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:03.119+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:03.454 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:03.453+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:04.076 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:04.075+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:04.460 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:04.458+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:05.036 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:05.034+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:05.491 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:05.490+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:05.991 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:05.990+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:06.532 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:06.531+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:06.986 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:06.985+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:07.574 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:07.573+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:08.012 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:08.010+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:08.593 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:08.591+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:08.973 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:08.972+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:09.619 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:09.618+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:09.926 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:09.925+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:10.596 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:10.595+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:10.948 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:10.947+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:11.641 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:11.640+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:11.979 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:11.978+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:12.658 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:12.657+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:12.982 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:12.981+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:13.610 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:13.608+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:13.952 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:13.951+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:14.573 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:14.572+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:14.930 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:14.928+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:15.591 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:15.590+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:15.891 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:15.890+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:16.545 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:16.543+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:16.844 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:16.842+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:17.578 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:17.577+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:17.843 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:17.842+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:18.605 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:18.603+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:18.887 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:18.886+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:19.568 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:19.567+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:19.864 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:19.863+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:20.530 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:20.529+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:20.852 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:20.850+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:21.490 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:21.489+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:21.901 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:21.900+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:22.464 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:22.463+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:22.908 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:22.906+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:23.503 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:23.502+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:23.874 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:23.873+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:24.475 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:24.474+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:24.848 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:24.846+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:25.447 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:25.445+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:25.893 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:25.892+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:26.459 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:26.458+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:26.905 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:26.903+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:27.479 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:27.478+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:27.868 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:27.867+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:28.496 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:28.495+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:28.839 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:28.838+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:29.485 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:29.484+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:29.826 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:29.825+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:30.515 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:30.514+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:30.815 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:30.813+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:31.494 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:31.493+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:31.832 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:31.830+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:32.465 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:32.464+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:32.871 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:32.869+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:33.459 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:33.457+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:33.868 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:33.868+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:34.440 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:34.438+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:34.844 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:34.843+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:35.418 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:35.417+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:35.800 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:35.799+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:36.379 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:36.379+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:36.847 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:36.847+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:37.334 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:37.334+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:37.825 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:37.821+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:38.305 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:38.305+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:38.849 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:38.848+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:39.277 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:39.277+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:39.810 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:39.810+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:40.302 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:40.302+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:40.811 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:40.811+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:41.255 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:41.255+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:41.853 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:41.852+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:42.231 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:42.230+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:42.837 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:42.837+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:43.200 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:43.200+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:43.790 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:43.790+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:44.209 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:44.209+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:44.784 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:44.784+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:45.256 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:45.256+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:45.792 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:45.791+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:46.246 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:46.246+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:46.838 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:46.838+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:47.248 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:47.248+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:47.810 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:47.810+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:48.292 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:48.292+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:48.844 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:48.844+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:49.245 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:49.245+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:49.838 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:49.838+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:50.268 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:50.268+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:50.819 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:50.819+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:51.248 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:51.247+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:51.826 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:51.826+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:52.214 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:52.214+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:52.854 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:52.854+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:53.252 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:53.250+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:53.901 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:53.900+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:54.237 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:54.236+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:54.877 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:54.875+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:55.268 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:55.267+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:55.912 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:55.910+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:56.313 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:56.312+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:56.957 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:56.956+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:57.298 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:57.296+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:57.977 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:57.976+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:58.286 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:58.285+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:58.951 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:58.949+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:59.324 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:21:59.322+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:21:59.950 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:21:59.948+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:00.345 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:00.344+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:00.961 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:00.960+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:01.347 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:01.346+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:01.989 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:01.988+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:02.311 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:02.310+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:03.015 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:03.014+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:03.270 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:03.269+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:04.014 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:04.013+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:04.256 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:04.255+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:04.972 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:04.971+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:05.221 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:05.220+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:06.006 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:06.004+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:06.223 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:06.222+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:07.026 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:07.024+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:07.220 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:07.219+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:08.037 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:08.036+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:08.211 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:08.210+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:09.048 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:09.047+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:09.220 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:09.219+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:10.037 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:10.035+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:10.180 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:10.179+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:11.051 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:11.050+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:11.142 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:11.141+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:12.032 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:12.031+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:12.133 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:12.131+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:13.040 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:13.038+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:13.093 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:13.092+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:14.011 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:14.009+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:14.144 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:14.142+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:14.973 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:14.972+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:15.177 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:15.176+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:15.992 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:15.991+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:16.220 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:16.219+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:17.002 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:17.001+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:17.226 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:17.225+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:18.035 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:18.034+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:18.259 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:18.258+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:19.047 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:19.046+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:19.280 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:19.278+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:20.008 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:20.006+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:20.302 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:20.301+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:21.042 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:21.040+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:21.347 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:21.346+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:22.086 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:22.085+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:22.378 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:22.377+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:23.060 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:23.059+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:23.355 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:23.354+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:24.051 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:24.050+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:24.358 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:24.357+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:25.020 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:25.018+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:25.375 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:25.373+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:26.029 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:26.029+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:26.395 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:26.395+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:27.068 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:27.067+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:27.423 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:27.423+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:28.023 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:28.023+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:28.393 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:28.392+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:29.061 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:29.061+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:29.386 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:29.386+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:30.048 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:30.047+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:30.429 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:30.429+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:31.040 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:31.040+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:31.461 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:31.461+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:32.040 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:32.039+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:32.485 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:32.484+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:33.087 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:33.087+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:33.503 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:33.503+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:34.047 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:34.047+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:34.469 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:34.469+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:35.080 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:35.079+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:35.510 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:35.509+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:36.080 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:36.080+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:36.471 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:36.471+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:37.115 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:37.114+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:37.495 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:37.494+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:38.127 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:38.127+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:38.518 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:38.518+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:39.110 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:39.110+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:39.538 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:39.538+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:40.143 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:40.143+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:40.503 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:40.503+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:41.137 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:41.137+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:41.479 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:41.478+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:42.127 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:42.127+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:42.505 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:42.505+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:43.164 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:43.163+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:43.461 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:43.460+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:44.155 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:44.153+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:44.491 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:44.490+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:45.187 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:45.185+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:45.513 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:45.512+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:46.171 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:46.170+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:46.475 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:46.474+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:47.221 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:47.217+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:47.492 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:47.491+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:48.194 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:48.193+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:48.505 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:48.504+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:49.207 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:49.206+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:49.545 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:49.543+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:50.184 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:50.182+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:50.498 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:50.496+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:51.149 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:51.147+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:51.547 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:51.546+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:52.105 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:52.104+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:52.534 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:52.532+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:53.079 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:53.078+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:53.569 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:53.568+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:54.085 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:54.084+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:54.594 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:54.593+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:55.076 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:55.074+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:55.602 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:55.600+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:56.087 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:56.085+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:56.642 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:56.641+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:57.086 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:57.085+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:57.689 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:57.688+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:58.097 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:58.096+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:58.657 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:58.656+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:59.079 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:22:59.077+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:22:59.612 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:22:59.611+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:00.086 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:00.085+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:00.607 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:00.605+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:01.119 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:01.118+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:01.637 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:01.635+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:02.078 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:02.076+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:02.619 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:02.618+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:03.038 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:03.036+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:03.663 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:03.662+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:04.020 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:04.018+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:04.709 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:04.708+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:05.026 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:05.025+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:05.741 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:05.739+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:06.043 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:06.042+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:06.778 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:06.777+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:07.053 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:07.051+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:07.789 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:07.788+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:08.037 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:08.036+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:08.760 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:08.759+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:09.083 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:09.082+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:09.732 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:09.731+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:10.074 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:10.073+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:10.739 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:10.738+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:11.028 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:11.027+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:11.695 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:11.694+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:12.052 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:12.051+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:12.718 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:12.717+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:13.022 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:13.021+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:13.674 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:13.673+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:14.027 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:14.026+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:14.709 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:14.708+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:15.038 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:15.037+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:15.699 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:15.698+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:16.084 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:16.084+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:16.650 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:16.650+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:17.070 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:17.069+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:17.700 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:17.700+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:18.021 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:18.021+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:18.689 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:18.689+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:19.015 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:19.015+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:19.708 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:19.708+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:19.988 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:19.988+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:20.696 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:20.696+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:21.005 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:21.005+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:21.730 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:21.730+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:21.995 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:21.994+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:22.747 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:22.747+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:22.987 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:22.987+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:23.797 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:23.797+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:23.938 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:23.938+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:24.812 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:24.812+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:24.947 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:24.947+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:25.835 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:25.835+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:25.996 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:25.995+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:26.816 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:26.815+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:27.000 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:27.000+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:27.816 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:27.816+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:27.995 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:27.995+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:28.804 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:28.804+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:28.979 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:28.979+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:29.785 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:29.785+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:29.940 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:29.939+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:30.811 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:30.810+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:30.980 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:30.979+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:31.816 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:31.816+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:32.004 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:32.004+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:32.808 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:32.808+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:33.031 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:33.031+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:33.839 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:33.837+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:34.076 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:34.075+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:34.797 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:34.795+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:35.032 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:35.031+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:35.827 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:35.825+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:36.066 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:36.065+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:36.803 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:36.802+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:37.046 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:37.045+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:37.812 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:37.811+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:38.075 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:38.074+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:38.796 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:38.795+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:39.036 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:39.035+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:39.827 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:39.826+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:39.996 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:39.994+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:40.852 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:40.851+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:41.046 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:41.045+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:41.813 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:41.812+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:42.083 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:42.082+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:42.828 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:42.827+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:43.096 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:43.094+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:43.809 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:43.807+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:44.075 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:44.074+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:44.805 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:44.804+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:45.120 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:45.119+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:45.830 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:45.829+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:46.098 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:46.096+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:46.840 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:46.839+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:47.080 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:47.079+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:47.823 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:47.822+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:48.054 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:48.053+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:48.840 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:48.839+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:49.098 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:49.097+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:49.829 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:49.827+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:50.055 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:50.054+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:50.846 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:50.845+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:51.070 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:51.069+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:51.845 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:51.843+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:52.100 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:52.098+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:52.818 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:52.817+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:53.128 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:53.127+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:53.817 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:53.816+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:54.104 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:54.103+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:54.826 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:54.824+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:55.116 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:55.114+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:55.859 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:55.857+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:56.082 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:56.081+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:56.840 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:56.839+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:57.124 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:57.123+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:57.846 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:57.845+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:58.137 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:58.136+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:58.815 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:58.813+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:59.111 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:23:59.110+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:23:59.814 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:23:59.812+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:00.122 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:00.121+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:00.844 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:00.843+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:01.119 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:01.118+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:01.868 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:01.867+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:02.152 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:02.151+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:02.837 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:02.835+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:03.202 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:03.200+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:03.848 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:03.847+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:04.201 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:04.201+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:04.860 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:04.859+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:05.219 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:05.218+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:05.831 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:05.830+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:06.213 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:06.213+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:06.848 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:06.848+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:07.253 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:07.253+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:07.827 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:07.827+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:08.290 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:08.290+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:08.781 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:08.780+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:09.294 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:09.294+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:09.767 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:09.767+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:10.276 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:10.276+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:10.793 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:10.793+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:11.316 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:11.315+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:11.833 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:11.833+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:12.305 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:12.305+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:12.800 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:12.800+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:13.308 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:13.308+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:13.807 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:13.807+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:14.297 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:14.297+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:14.841 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:14.841+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:15.343 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:15.342+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:15.842 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:15.842+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:16.393 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:16.393+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:16.810 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:16.810+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:17.353 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:17.352+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:17.833 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:17.833+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:18.332 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:18.332+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:18.790 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:18.789+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:19.367 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:19.367+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:19.770 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:19.770+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:20.404 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:20.403+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:20.759 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:20.759+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:21.386 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:21.386+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:21.808 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:21.808+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:22.345 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:22.344+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:22.799 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:22.798+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:23.340 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:23.339+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:23.763 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:23.762+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:24.316 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:24.315+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:24.793 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:24.792+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:25.325 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:25.324+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:25.773 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:25.772+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:26.340 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:26.339+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:26.818 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:26.816+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:27.344 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:27.343+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:27.797 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:27.796+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:28.347 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:28.346+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:28.784 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:28.782+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:29.393 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:29.391+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:29.780 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:29.778+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:30.419 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:30.418+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:30.745 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:30.744+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:31.465 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:31.464+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:31.789 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:31.788+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:32.438 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:32.437+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:32.797 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:32.796+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:33.451 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:33.450+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:33.848 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:33.846+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:34.418 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:34.417+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:34.884 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:34.883+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:35.462 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:35.461+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:35.848 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:35.847+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:36.495 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:36.494+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:36.864 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:36.863+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:37.482 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:37.481+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:37.909 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:37.908+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:38.435 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:38.434+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:38.909 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:38.908+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:39.477 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:39.475+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:39.928 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:39.927+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:40.495 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:40.494+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:40.971 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:40.970+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:41.543 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:41.541+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:42.015 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:42.014+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:42.552 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:42.551+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:43.005 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:43.004+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:43.504 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:43.503+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:44.017 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:44.016+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:44.546 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:44.545+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:45.020 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:45.018+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:45.505 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:45.504+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:46.051 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:46.050+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:46.487 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:46.485+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:47.041 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:47.039+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:47.470 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:47.468+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:48.068 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:48.067+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:48.470 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:48.469+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:49.047 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:49.046+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:49.453 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:49.452+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:50.019 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:50.018+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:50.494 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:50.492+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:51.036 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:51.034+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:51.479 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:51.478+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:52.044 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:52.043+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:52.512 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:52.510+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:53.087 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:53.085+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:53.518 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:53.516+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:54.089 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:54.088+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:54.540 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:54.538+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:55.127 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:55.126+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:55.568 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:55.567+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:56.131 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:56.131+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:56.521 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:56.520+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:57.151 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:57.151+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:57.547 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:57.547+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:58.179 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:58.179+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:58.534 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:58.534+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:59.203 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:24:59.203+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:24:59.562 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:24:59.561+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:00.198 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:00.198+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:00.525 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:00.525+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:01.195 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:01.195+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:01.493 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:01.492+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:02.217 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:02.217+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:02.514 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:02.514+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:03.204 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:03.204+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:03.561 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:03.561+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:04.207 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:04.206+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:04.557 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:04.557+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:05.201 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:05.201+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:05.560 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:05.559+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:06.196 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:06.195+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:06.515 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:06.514+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:07.153 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:07.153+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:07.521 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:07.520+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:08.136 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:08.135+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:08.499 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:08.499+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:09.160 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:09.160+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:09.547 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:09.547+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:10.141 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:10.141+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:10.566 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:10.566+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:11.117 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:11.116+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:11.543 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:11.543+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:12.157 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:12.156+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:12.542 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:12.542+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:13.173 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:13.173+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:13.513 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:13.512+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:14.212 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:14.210+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:14.509 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:14.507+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:15.227 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:15.225+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:15.505 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:15.504+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:16.193 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:16.191+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:16.507 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:16.506+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:17.188 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:17.187+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:17.496 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:17.495+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:18.179 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:18.178+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:18.455 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:18.454+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:19.152 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:19.150+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:19.469 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:19.468+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:20.139 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:20.138+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:20.443 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:20.442+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:21.135 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:21.134+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:21.466 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:21.465+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:22.129 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:22.128+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:22.499 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:22.498+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:23.099 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:23.097+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:23.511 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:23.510+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:24.075 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:24.074+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:24.470 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:24.468+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:25.080 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:25.078+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:25.508 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:25.507+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:26.088 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:26.087+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:26.521 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:26.520+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:27.064 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:27.062+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:27.480 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:27.478+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:28.044 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:28.042+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:28.510 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:28.509+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:29.023 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:29.021+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:29.552 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:29.550+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:29.996 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:29.995+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:30.505 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:30.504+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:31.032 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:31.031+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:31.476 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:31.475+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:32.012 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:32.011+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:32.527 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:32.526+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:33.005 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:33.003+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:33.555 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:33.553+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:34.026 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:34.025+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:34.587 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:34.586+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:34.995 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:34.994+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:35.573 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:35.572+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:36.004 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:36.003+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:36.539 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:36.537+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:37.013 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:37.012+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:37.541 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:37.540+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:38.025 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:38.023+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:38.572 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:38.571+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:39.040 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:39.038+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:39.610 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:39.608+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:39.995 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:39.993+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:40.570 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:40.569+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:41.043 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:41.041+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:41.598 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:41.597+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:41.998 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:41.997+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:42.609 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:42.608+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:43.016 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:43.015+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:43.560 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:43.559+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:44.005 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:44.004+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:44.529 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:44.527+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:44.966 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:44.965+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:45.532 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:45.531+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:45.986 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:45.985+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:46.513 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:46.512+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:46.983 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:46.983+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:47.510 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:47.509+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:47.959 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:47.959+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:48.495 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:48.495+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:48.970 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:48.970+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:49.517 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:49.517+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:49.941 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:49.941+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:50.520 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:50.520+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:50.935 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:50.935+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:51.567 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:51.567+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:51.952 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:51.952+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:52.541 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:52.541+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:52.912 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:52.912+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:53.572 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:53.571+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:53.874 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:53.874+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:54.608 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:54.608+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:54.843 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:54.843+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:55.560 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:55.560+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:55.856 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:55.856+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:56.566 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:56.566+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:56.889 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:56.889+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:57.544 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:57.544+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:57.874 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:57.873+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:58.541 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:58.541+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:58.840 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:58.840+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:59.509 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:25:59.508+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:25:59.843 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:25:59.843+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:00.466 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:00.465+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:00.808 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:00.807+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:01.477 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:01.477+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:01.806 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:01.806+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:02.450 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:02.450+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:02.774 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:02.774+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:03.483 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:03.482+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:03.753 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:03.752+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:04.504 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:04.503+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:04.746 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:04.745+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:05.532 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:05.531+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:05.752 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:05.750+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:06.548 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:06.546+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:06.802 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:06.800+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:07.557 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:07.556+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:07.765 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:07.764+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:08.548 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:08.547+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:08.781 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:08.780+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:09.584 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:09.583+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:09.830 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:09.829+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:10.621 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:10.619+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:10.848 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:10.847+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:11.608 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:11.607+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:11.815 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:11.813+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:12.638 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:12.637+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:12.774 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:12.773+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:13.670 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:13.669+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:13.754 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:13.753+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:14.656 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:14.655+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:14.770 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:14.769+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:15.632 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:15.631+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:15.793 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:15.792+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:16.646 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:16.644+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:16.830 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:16.829+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:17.600 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:17.599+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:17.853 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:17.851+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:18.572 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:18.571+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:18.841 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:18.840+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:19.542 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:19.541+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:19.819 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:19.818+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:20.570 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:20.569+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:20.849 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:20.847+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:21.548 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:21.547+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:21.811 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:21.810+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:22.555 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:22.554+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:22.849 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:22.848+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:23.566 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:23.565+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:23.816 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:23.815+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:24.540 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:24.539+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:24.787 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:24.786+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:25.517 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:25.516+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:25.837 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:25.836+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:26.538 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:26.537+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:26.886 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:26.884+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:27.490 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:27.489+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:27.840 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:27.839+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:28.452 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:28.451+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:28.844 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:28.843+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:29.449 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:29.448+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:29.871 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:29.870+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:30.427 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:30.425+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:30.898 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:30.897+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:31.394 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:31.393+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:31.915 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:31.913+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:32.358 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:32.356+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:32.961 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:32.960+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:33.405 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:33.404+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:33.970 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:33.969+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:34.430 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:34.429+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:34.977 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:34.976+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:35.400 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:35.399+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:36.026 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:36.026+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:36.447 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:36.446+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:36.997 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:36.997+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:37.401 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:37.401+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:38.029 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:38.029+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:38.420 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:38.420+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:39.006 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:39.005+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:39.424 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:39.424+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:40.033 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:40.033+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:40.444 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:40.443+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:41.058 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:41.058+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:41.446 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:41.445+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:42.016 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:42.015+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:42.491 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:42.491+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:42.978 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:42.978+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:43.450 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:43.449+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:43.992 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:43.991+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:44.460 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:44.459+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:45.040 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:45.040+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:45.506 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:45.506+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:46.029 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:46.029+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:46.516 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:46.516+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:47.006 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:47.006+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:47.480 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:47.479+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:47.986 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:47.986+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:48.466 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:48.466+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:49.008 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:49.008+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:49.425 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:49.425+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:50.021 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:50.018+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:50.384 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:50.383+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:50.978 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:50.977+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:51.357 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:51.357+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:52.028 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:52.027+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:52.322 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:52.322+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:53.047 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:53.046+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:53.354 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:53.352+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:54.083 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:54.081+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:54.360 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:54.359+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:55.067 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:55.065+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:55.362 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:55.360+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:56.042 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:56.040+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:56.379 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:56.377+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:57.013 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:57.012+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:57.360 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:57.359+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:57.984 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:57.983+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:58.399 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:58.398+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:58.990 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:26:58.988+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:26:59.364 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:26:59.362+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:00.002 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:00.001+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:00.339 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:00.338+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:01.017 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:01.016+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:01.382 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:01.381+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:02.018 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:02.017+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:02.414 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:02.413+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:02.989 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:02.988+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:03.387 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:03.385+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:03.990 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:03.988+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:04.355 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:04.354+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:04.942 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:04.941+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:05.343 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:05.341+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:05.992 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:05.991+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:06.390 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:06.388+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:06.986 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:06.985+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:07.387 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:07.385+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:07.949 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:07.948+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:08.372 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:08.371+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:08.919 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:08.917+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:09.403 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:09.401+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:09.874 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:09.873+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:10.363 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:10.362+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:10.918 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:10.917+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:11.345 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:11.344+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:11.944 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:11.943+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:12.297 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:12.296+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:12.954 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:12.953+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:13.291 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:13.290+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:13.976 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:13.974+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:14.278 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:14.277+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:14.994 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:14.992+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:15.231 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:15.229+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:15.950 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:15.948+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:16.220 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:16.219+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:16.952 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:16.950+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:17.226 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:17.225+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:17.937 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:17.935+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:18.218 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:18.217+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:18.963 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:18.962+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:19.204 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:19.203+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:19.978 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:19.977+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:20.253 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:20.252+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:20.952 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:20.951+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:21.302 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:21.300+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:21.984 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:21.983+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:22.260 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:22.259+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:22.936 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:22.935+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:23.263 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:23.262+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:23.964 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:23.963+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:24.243 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:24.242+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:24.916 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:24.915+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:25.255 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:25.253+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:25.958 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:25.957+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:26.214 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:26.214+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:26.936 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:26.936+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:27.243 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:27.243+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:27.943 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:27.943+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:28.249 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:28.249+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:28.929 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:28.929+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:29.201 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:29.201+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:29.970 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:29.970+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:30.206 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:30.205+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:30.972 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:30.972+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:31.221 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:31.220+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:31.968 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:31.967+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:32.206 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:32.205+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:32.982 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:32.982+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:33.161 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:33.160+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:33.961 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:33.961+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:34.174 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:34.174+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:34.925 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:34.924+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:35.175 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:35.175+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:35.929 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:35.929+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:36.180 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:36.179+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:36.972 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:36.972+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:37.197 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:37.197+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:38.022 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:38.022+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:38.166 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:38.166+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:39.046 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:39.045+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:39.126 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:39.125+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:40.021 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:40.020+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:40.104 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:40.104+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:41.048 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:41.047+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:41.076 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:41.076+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:42.012 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:42.011+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:42.048 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:42.048+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:42.970 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:42.970+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:43.015 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:43.014+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:43.955 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:43.953+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:44.003 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:44.002+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:44.946 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:44.945+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:44.977 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:44.976+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:45.943 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:45.941+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:45.943 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:45.941+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:46.979 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:46.977+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:46.992 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:46.991+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:47.956 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:47.954+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:48.024 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:48.023+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:48.932 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:48.930+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:49.051 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:49.050+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:49.961 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:49.960+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:50.064 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:50.062+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:50.975 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:50.974+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:51.095 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:51.094+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:51.968 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:51.966+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:52.127 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:52.126+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:52.968 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:52.966+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:53.117 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:53.116+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:53.991 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:53.990+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:54.078 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:54.076+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:54.986 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:54.985+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:55.031 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:55.029+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:56.031 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:56.030+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:56.051 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:56.050+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:57.014 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:57.011+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:57.028 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:57.027+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:58.054 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:58.053+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:58.068 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:58.067+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:59.086 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:27:59.085+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:27:59.100 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:27:59.099+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:00.063 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:00.061+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:00.114 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:00.113+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:01.088 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:01.087+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:01.110 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:01.109+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:02.046 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:02.044+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:02.090 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:02.089+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:03.014 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:03.013+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:03.094 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:03.093+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:04.062 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:04.061+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:04.137 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:04.136+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:05.020 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:05.018+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:05.149 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:05.148+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:06.053 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:06.052+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:06.102 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:06.101+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:07.012 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:07.011+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:07.116 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:07.115+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:08.022 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:08.021+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:08.137 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:08.136+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:09.051 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:09.050+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:09.151 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:09.149+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:10.079 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:10.078+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:10.145 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:10.144+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:11.068 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:11.067+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:11.184 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:11.183+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:12.034 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:12.033+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:12.178 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:12.177+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:13.079 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:13.077+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:13.170 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:13.169+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:14.116 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:14.115+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:14.194 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:14.193+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:15.076 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:15.075+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:15.214 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:15.214+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:16.063 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:16.062+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:16.180 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:16.180+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:17.031 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:17.031+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:17.222 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:17.222+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:18.077 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:18.076+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:18.254 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:18.254+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:19.124 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:19.124+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:19.298 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:19.297+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:20.079 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:20.079+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:20.297 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:20.297+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:21.077 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:21.076+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:21.276 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:21.276+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:22.126 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:22.126+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:22.293 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:22.293+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:23.111 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:23.111+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:23.244 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:23.244+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:24.135 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:24.135+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:24.240 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:24.240+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:25.126 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:25.126+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:25.223 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:25.223+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:26.173 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:26.172+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:26.177 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:26.177+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:27.193 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:27.192+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:27.212 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:27.212+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:28.173 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:28.172+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:28.186 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:28.186+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:29.126 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:29.126+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:29.157 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:29.157+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:30.126 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:30.126+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:30.147 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:30.147+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:31.117 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:31.116+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:31.163 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:31.163+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:32.129 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:32.129+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:32.183 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:32.183+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:33.117 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:33.116+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:33.160 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:33.159+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:34.127 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:34.126+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:34.150 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:34.148+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:35.159 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:35.158+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:35.189 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:35.188+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:36.181 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:36.179+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:36.235 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:36.233+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:37.199 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:37.198+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:37.254 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:37.252+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:38.217 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:38.216+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:38.223 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:38.221+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:39.182 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:39.180+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:39.246 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:39.245+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:40.157 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:40.156+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:40.269 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:40.268+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:41.113 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:41.112+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:41.293 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:41.292+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:42.099 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:42.098+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:42.341 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:42.340+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:43.066 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:43.065+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:43.314 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:43.313+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:44.058 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:44.057+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:44.271 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:44.270+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:45.023 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:45.022+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:45.307 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:45.306+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:45.993 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:45.991+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:46.260 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:46.259+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:47.001 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:47.000+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:47.231 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:47.230+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:47.963 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:47.962+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:48.236 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:48.235+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:48.945 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:48.943+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:49.252 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:49.251+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:49.939 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:49.937+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:50.216 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:50.215+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:50.922 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:50.920+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:51.231 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:51.230+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:51.886 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:51.884+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:52.188 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:52.186+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:52.872 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:52.871+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:53.219 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:53.217+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:53.921 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:53.919+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:54.236 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:54.235+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:54.927 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:54.926+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:55.204 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:55.203+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:55.882 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:55.881+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:56.187 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:56.186+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:56.838 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:56.837+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:57.153 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:57.152+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:57.829 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:57.827+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:58.159 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:58.158+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:58.877 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:58.876+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:59.135 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:28:59.134+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:28:59.910 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:28:59.909+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:00.142 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:00.141+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:00.879 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:00.878+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:01.146 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:01.144+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:01.890 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:01.889+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:02.175 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:02.174+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:02.872 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:02.870+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:03.220 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:03.218+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:03.859 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:03.858+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:04.194 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:04.193+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:04.865 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:04.864+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:05.202 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:05.200+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:05.838 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:05.838+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:06.216 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:06.215+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:06.873 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:06.873+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:07.173 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:07.172+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:07.858 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:07.857+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:08.143 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:08.143+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:08.879 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:08.878+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:09.187 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:09.187+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:09.875 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:09.874+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:10.225 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:10.225+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:10.858 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:10.857+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:11.229 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:11.229+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:11.877 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:11.876+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:12.251 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:12.251+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:12.865 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:12.864+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:13.266 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:13.265+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:13.825 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:13.825+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:14.286 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:14.286+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:14.799 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:14.799+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:15.323 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:15.323+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:15.759 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:15.759+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:16.336 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:16.336+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:16.788 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:16.788+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:17.323 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:17.323+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:17.760 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:17.759+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:18.320 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:18.319+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:18.713 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:18.713+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:19.363 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:19.363+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:19.749 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:19.749+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:20.327 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:20.327+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:20.789 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:20.788+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:21.364 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:21.364+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:21.754 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:21.754+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:22.332 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:22.332+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:22.759 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:22.759+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:23.376 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:23.374+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:23.753 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:23.752+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:24.402 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:24.401+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:24.758 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:24.757+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:25.380 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:25.379+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:25.740 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:25.738+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:26.384 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:26.382+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:26.733 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:26.732+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:27.381 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:27.380+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:27.770 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:27.769+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:28.361 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:28.360+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:28.756 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:28.754+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:29.383 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:29.382+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:29.724 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:29.723+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:30.376 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:30.375+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:30.710 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:30.708+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:31.335 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:31.334+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:31.737 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:31.736+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:32.335 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:32.334+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:32.754 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:32.753+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:33.364 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:33.362+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:33.789 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:33.788+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:34.335 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:34.334+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:34.803 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:34.802+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:35.312 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:35.311+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:35.783 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:35.782+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:36.363 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:36.362+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:36.748 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:36.747+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:37.409 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:37.407+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:37.778 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:37.776+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:38.400 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:38.398+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:38.762 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:38.761+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:39.390 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:39.389+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:39.740 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:39.739+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:40.389 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:40.387+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:40.755 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:40.754+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:41.346 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:41.343+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:41.769 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:41.768+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:42.340 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:42.339+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:42.784 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:42.783+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:43.374 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:43.372+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:43.807 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:43.806+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:44.386 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:44.385+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:44.780 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:44.778+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:45.344 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:45.343+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:45.750 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:45.748+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:46.343 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:46.342+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:46.797 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:46.796+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:47.357 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:47.356+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:47.818 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:47.817+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:48.319 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:48.318+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:48.801 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:48.800+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:49.332 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:49.330+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:49.850 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:49.848+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:50.367 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:50.365+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:50.880 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:50.879+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:51.339 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:51.338+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:51.925 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:51.924+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:52.344 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:52.343+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:52.948 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:52.947+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:53.385 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:53.384+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:53.952 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:53.952+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:54.425 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:54.424+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:54.949 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:54.948+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:55.451 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:55.451+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:55.950 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:55.950+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:56.451 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:56.450+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:56.926 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:56.925+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:57.453 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:57.453+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:57.899 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:57.899+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:58.432 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:58.431+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:58.898 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:58.897+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:59.438 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:29:59.438+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:29:59.890 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:29:59.889+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:00.458 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:00.458+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:00.923 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:00.922+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:01.426 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:01.426+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:01.954 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:01.953+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:02.383 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:02.383+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:02.966 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:02.966+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:03.374 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:03.373+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:04.010 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:04.009+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:04.356 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:04.356+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:05.040 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:05.039+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:05.406 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:05.406+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:06.008 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:06.008+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:06.428 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:06.428+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:06.978 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:06.977+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:07.448 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:07.448+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:07.930 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:07.929+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:08.456 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:08.455+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:08.915 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:08.914+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:09.475 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:09.475+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:09.886 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:09.886+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:10.488 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:10.488+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:10.895 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:10.894+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:11.498 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:11.498+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:11.848 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:11.848+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:12.514 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:12.514+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:12.861 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:12.860+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:13.505 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:13.503+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:13.819 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:13.818+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:14.501 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:14.500+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:14.794 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:14.793+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:15.479 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:15.477+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:15.819 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:15.817+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:16.510 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:16.509+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:16.815 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:16.814+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:17.521 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:17.520+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:17.829 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:17.828+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:18.495 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:18.494+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:18.832 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:18.831+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:19.476 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:19.474+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:19.850 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:19.848+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:20.480 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:20.479+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:20.831 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:20.830+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:21.482 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:21.481+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:21.794 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:21.793+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:22.515 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:22.514+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:22.787 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:22.785+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:23.504 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:23.502+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:23.755 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:23.754+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:24.550 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:24.549+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:24.770 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:24.769+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:25.571 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:25.570+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:25.781 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:25.779+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:26.529 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:26.527+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:26.778 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:26.777+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:27.557 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:27.555+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:27.786 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:27.785+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:28.563 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:28.562+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:28.790 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:28.789+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:29.541 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:29.540+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:29.805 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:29.804+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:30.571 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:30.570+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:30.795 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:30.794+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:31.550 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:31.549+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:31.828 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:31.827+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:32.569 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:32.568+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:32.826 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:32.824+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:33.520 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:33.519+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:33.844 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:33.843+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:34.546 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:34.545+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:34.836 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:34.835+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:35.545 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:35.543+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:35.811 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:35.809+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:36.508 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:36.506+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:36.814 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:36.813+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:37.515 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:37.514+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:37.830 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:37.828+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:38.485 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:38.484+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:38.872 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:38.870+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:39.450 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:39.449+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:39.840 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:39.838+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:40.470 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:40.469+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:40.885 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:40.884+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:41.429 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:41.427+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:41.884 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:41.883+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:42.468 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:42.467+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:42.928 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:42.927+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:43.504 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:43.502+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:43.978 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:43.977+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:44.536 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:44.535+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:45.001 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:44.999+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:45.510 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:45.510+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:46.038 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:46.038+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:46.557 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:46.557+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:47.042 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:47.041+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:47.544 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:47.544+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:48.022 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:48.021+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:48.587 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:48.587+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:48.998 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:48.997+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:49.620 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:49.620+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:50.023 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:50.023+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:50.609 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:50.608+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:51.015 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:51.015+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:51.623 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:51.623+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:52.063 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:52.062+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:52.645 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:52.645+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:53.099 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:53.098+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:53.604 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:53.604+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:54.099 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:54.099+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:54.555 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:54.555+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:55.091 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:55.091+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:55.608 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:55.606+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:56.128 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:56.128+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:56.636 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:56.636+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:57.171 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:57.171+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:57.654 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:57.653+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:58.203 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:58.203+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:58.651 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:58.651+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:59.187 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:30:59.187+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:30:59.635 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:30:59.635+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:00.149 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:00.148+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:00.653 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:00.653+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:01.102 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:01.101+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:01.691 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:01.691+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:02.114 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:02.114+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:02.724 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:02.722+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:03.089 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:03.088+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:03.735 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:03.734+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:04.133 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:04.131+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:04.716 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:04.714+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:05.172 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:05.171+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:05.672 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:05.670+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:06.145 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:06.144+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:06.700 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:06.698+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:07.102 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:07.101+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:07.737 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:07.735+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:08.097 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:08.096+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:08.754 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:08.753+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:09.142 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:09.140+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:09.798 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:09.796+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:10.149 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:10.147+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:10.811 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:10.809+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:11.099 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:11.098+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:11.821 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:11.819+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:12.119 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:12.118+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:12.808 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:12.807+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:13.101 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:13.099+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:13.858 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:13.856+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:14.059 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:14.058+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:14.843 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:14.842+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:15.104 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:15.102+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:15.847 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:15.846+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:16.107 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:16.106+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:16.851 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:16.850+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:17.125 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:17.124+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:17.828 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:17.826+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:18.160 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:18.159+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:18.789 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:18.788+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:19.196 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:19.195+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:19.743 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:19.742+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:20.174 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:20.172+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:20.698 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:20.696+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:21.129 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:21.128+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:21.745 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:21.744+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:22.155 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:22.154+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:22.761 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:22.760+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:23.132 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:23.130+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:23.721 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:23.719+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:24.129 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:24.127+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:24.765 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:24.764+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:25.179 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:25.178+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:25.785 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:25.784+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:26.226 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:26.224+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:26.821 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:26.820+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:27.274 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:27.273+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:27.787 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:27.786+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:28.288 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:28.286+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:28.816 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:28.815+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:29.337 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:29.336+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:29.855 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:29.853+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:30.343 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:30.342+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:30.824 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:30.822+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:31.314 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:31.312+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:31.797 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:31.796+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:32.337 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:32.335+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:32.827 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:32.826+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:33.305 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:33.304+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:33.815 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:33.814+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:34.329 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:34.328+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:34.843 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:34.842+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:35.295 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:35.295+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:35.889 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:35.889+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:36.305 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:36.304+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:36.899 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:36.899+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:37.313 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:37.313+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:37.859 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:37.859+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:38.323 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:38.323+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:38.836 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:38.835+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:39.365 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:39.365+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:39.811 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:39.811+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:40.344 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:40.344+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:40.791 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:40.790+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:41.343 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:41.343+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:41.757 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:41.757+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:42.339 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:42.339+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:42.750 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:42.750+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:43.313 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:43.313+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:43.733 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:43.732+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:44.307 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:44.306+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:44.740 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:44.740+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:45.286 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:45.286+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:45.730 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:45.730+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:46.309 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:46.308+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:46.741 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:46.740+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:47.295 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:47.295+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:47.740 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:47.740+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:48.287 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:48.286+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:48.757 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:48.757+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:49.254 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:49.254+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:49.775 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:49.774+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:50.231 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:50.231+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:50.765 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:50.764+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:51.243 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:51.242+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:51.794 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:51.794+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:52.269 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:52.268+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:52.841 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:52.839+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:53.226 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:53.225+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:53.830 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:53.829+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:54.247 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:54.246+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:54.810 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:54.808+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:55.261 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:55.260+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:55.855 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:55.854+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:56.288 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:56.286+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:56.872 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:56.870+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:57.328 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:57.326+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:57.853 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:57.851+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:58.346 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:58.345+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:58.816 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:58.815+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:59.366 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:31:59.364+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:31:59.817 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:31:59.815+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:00.329 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:00.328+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:00.769 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:00.767+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:01.365 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:01.363+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:01.778 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:01.777+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:02.329 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:02.327+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:02.797 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:02.796+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:03.331 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:03.329+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:03.791 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:03.789+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:04.342 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:04.341+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:04.805 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:04.803+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:05.361 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:05.359+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:05.824 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:05.823+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:06.369 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:06.368+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:06.793 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:06.791+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:07.327 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:07.326+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:07.789 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:07.788+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:08.300 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:08.298+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:08.775 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:08.774+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:09.343 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:09.341+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:09.818 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:09.817+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:10.392 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:10.390+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:10.865 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:10.864+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:11.423 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:11.421+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:11.852 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:11.850+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:12.456 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:12.455+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:12.815 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:12.814+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:13.470 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:13.469+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:13.766 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:13.765+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:14.479 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:14.477+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:14.785 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:14.784+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:15.448 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:15.447+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:15.815 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:15.814+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:16.402 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:16.401+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:16.777 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:16.776+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:17.402 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:17.401+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:17.792 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:17.791+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:18.398 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:18.397+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:18.804 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:18.803+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:19.365 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:19.364+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:19.829 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:19.828+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:20.349 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:20.347+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:20.824 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:20.823+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:21.343 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:21.341+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:21.789 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:21.787+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:22.317 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:22.315+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:22.824 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:22.822+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:23.272 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:23.270+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:23.804 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:23.804+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:24.288 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:24.287+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:24.845 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:24.844+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:25.322 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:25.321+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:25.877 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:25.877+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:26.351 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:26.351+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:26.864 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:26.864+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:27.336 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:27.334+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:27.912 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:27.911+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:28.362 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:28.361+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:28.879 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:28.878+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:29.388 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:29.388+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:29.848 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:29.847+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:30.381 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:30.380+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:30.866 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:30.866+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:31.360 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:31.360+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:31.831 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:31.830+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:32.391 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:32.391+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:32.838 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:32.838+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:33.361 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:33.361+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:33.882 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:33.882+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:34.333 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:34.333+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:34.892 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:34.891+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:35.381 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:35.380+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:35.915 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:35.914+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:36.358 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:36.357+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:36.944 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:36.944+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:37.319 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:37.319+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:37.989 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:37.989+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:38.308 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:38.307+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:38.970 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:38.970+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:39.290 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:39.289+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:40.013 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:40.013+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:40.263 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:40.262+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:41.023 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:41.023+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:41.259 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:41.258+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:42.027 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:42.025+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:42.271 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:42.270+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:42.985 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:42.983+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:43.236 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:43.234+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:43.978 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:43.976+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:44.240 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:44.239+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:44.957 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:44.956+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:45.197 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:45.196+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:46.004 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:46.003+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:46.189 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:46.188+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:47.010 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:47.008+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:47.225 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:47.223+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:48.003 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:48.002+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:48.186 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:48.184+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:48.966 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:48.965+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:49.191 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:49.189+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:49.924 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:49.923+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:50.189 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:50.188+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:50.881 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:50.880+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:51.190 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:51.189+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:51.871 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:51.870+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:52.206 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:52.205+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:52.921 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:52.919+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:53.225 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:53.223+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:53.880 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:53.879+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:54.194 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:54.193+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:54.851 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:54.850+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:55.160 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:55.159+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:55.888 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:55.886+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:56.142 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:56.141+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:56.881 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:56.880+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:57.095 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:57.094+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:57.858 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:57.857+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:58.108 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:58.107+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:58.862 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:58.861+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:59.154 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:32:59.153+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:32:59.868 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:32:59.867+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:00.128 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:00.127+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:00.862 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:00.860+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:01.171 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:01.170+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:01.861 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:01.859+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:02.180 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:02.179+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:02.899 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:02.898+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:03.197 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:03.196+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:03.900 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:03.899+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:04.248 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:04.246+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:04.918 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:04.916+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:05.270 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:05.268+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:05.955 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:05.954+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:06.273 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:06.271+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:06.998 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:06.997+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:07.254 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:07.252+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:08.027 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:08.026+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:08.296 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:08.295+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:09.069 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:09.067+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:09.321 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:09.320+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:10.096 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:10.095+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:10.342 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:10.341+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:11.134 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:11.133+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:11.344 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:11.343+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:12.147 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:12.145+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:12.375 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:12.373+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:13.139 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:13.137+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:13.377 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:13.375+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:14.115 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:14.114+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:14.408 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:14.408+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:15.106 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:15.106+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:15.448 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:15.447+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:16.134 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:16.134+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:16.450 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:16.450+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:17.162 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:17.161+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:17.448 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:17.448+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:18.189 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:18.189+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:18.408 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:18.407+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:19.208 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:19.208+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:19.379 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:19.378+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:20.215 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:20.215+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:20.340 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:20.339+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:21.264 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:21.264+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:21.377 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:21.376+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:22.283 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:22.283+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:22.393 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:22.393+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:23.294 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:23.294+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:23.362 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:23.362+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:24.246 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:24.246+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:24.363 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:24.362+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:25.223 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:25.222+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:25.401 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:25.401+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:26.218 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:26.217+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:26.412 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:26.411+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:27.181 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:27.180+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:27.423 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:27.422+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:28.226 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:28.225+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:28.416 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:28.415+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:29.246 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:29.245+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:29.465 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:29.464+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:30.279 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:30.279+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:30.498 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:30.498+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:31.237 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:31.237+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:31.499 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:31.499+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:32.202 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:32.200+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:32.547 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:32.545+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:33.229 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:33.227+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:33.561 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:33.559+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:34.260 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:34.259+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:34.562 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:34.561+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:35.279 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:35.277+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:35.583 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:35.581+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:36.276 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:36.275+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:36.559 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:36.558+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:37.323 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:37.322+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:37.601 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:37.599+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:38.303 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:38.301+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:38.649 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:38.648+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:39.349 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:39.348+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:39.675 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:39.674+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:40.310 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:40.309+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:40.638 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:40.637+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:41.292 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:41.291+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:41.658 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:41.657+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:42.299 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:42.298+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:42.627 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:42.625+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:43.303 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:43.302+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:43.596 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:43.594+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:44.287 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:44.286+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:44.548 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:44.546+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:45.288 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:45.287+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:45.526 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:45.525+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:46.246 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:46.244+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:46.482 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:46.481+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:47.257 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:47.255+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:47.497 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:47.496+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:48.291 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:48.290+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:48.459 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:48.458+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:49.328 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:49.327+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:49.453 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:49.452+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:50.378 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:50.376+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:50.427 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:50.425+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:51.379 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:51.377+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:51.420 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:51.418+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:52.330 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:52.329+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:52.408 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:52.407+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:53.343 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:53.342+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:53.391 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:53.390+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:54.356 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:54.354+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:54.407 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:54.405+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:55.386 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:55.384+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:55.402 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:55.401+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:56.407 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:56.406+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:56.440 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:56.438+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:57.372 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:57.371+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:57.460 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:57.459+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:58.327 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:58.325+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:58.428 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:58.427+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:59.334 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:33:59.333+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:33:59.478 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:33:59.477+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:00.286 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:00.285+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:00.458 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:00.457+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:01.270 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:01.269+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:01.465 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:01.464+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:02.249 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:02.248+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:02.444 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:02.443+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:03.219 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:03.217+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:03.474 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:03.473+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:04.177 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:04.177+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:04.440 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:04.439+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:05.172 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:05.172+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:05.436 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:05.436+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:06.207 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:06.207+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:06.448 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:06.447+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:07.195 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:07.195+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:07.470 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:07.470+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:08.156 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:08.156+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:08.487 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:08.487+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:09.157 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:09.156+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:09.465 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:09.465+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:10.159 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:10.159+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:10.484 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:10.484+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:11.142 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:11.142+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:11.439 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:11.439+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:12.135 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:12.135+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:12.451 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:12.451+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:13.180 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:13.179+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:13.434 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:13.434+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:14.183 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:14.182+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:14.401 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:14.400+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:15.173 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:15.173+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:15.423 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:15.422+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:16.217 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:16.217+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:16.400 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:16.400+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:17.215 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:17.214+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:17.424 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:17.424+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:18.206 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:18.206+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:18.409 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:18.408+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:19.257 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:19.256+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:19.387 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:19.387+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:20.239 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:20.239+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:20.437 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:20.437+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:21.256 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:21.256+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:21.488 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:21.487+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:22.239 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:22.238+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:22.464 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:22.463+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:23.221 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:23.220+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:23.420 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:23.418+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:24.233 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:24.232+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:24.424 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:24.423+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:25.217 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:25.215+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:25.402 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:25.400+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:26.202 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:26.201+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:26.449 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:26.448+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:27.209 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:27.208+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:27.465 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:27.461+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:28.192 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:28.191+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:28.498 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:28.496+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:29.216 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:29.214+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:29.450 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:29.449+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:30.167 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:30.165+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:30.411 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:30.409+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:31.192 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:31.190+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:31.460 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:31.458+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:32.207 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:32.206+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:32.465 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:32.464+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:33.197 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:33.195+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:33.427 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:33.426+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:34.193 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:34.192+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:34.389 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:34.388+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:35.227 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:35.226+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:35.397 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:35.396+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:36.245 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:36.243+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:36.381 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:36.379+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:37.222 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:37.220+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:37.377 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:37.375+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:38.215 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:38.213+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:38.407 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:38.406+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:39.259 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:39.257+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:39.420 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:39.419+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:40.292 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:40.290+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:40.451 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:40.449+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:41.274 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:41.273+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:41.459 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:41.458+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:42.247 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:42.245+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:42.437 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:42.436+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:43.202 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:43.201+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:43.414 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:43.413+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:44.229 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:44.227+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:44.460 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:44.458+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:45.278 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:45.276+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:45.478 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:45.477+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:46.256 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:46.255+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:46.477 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:46.475+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:47.237 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:47.236+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:47.468 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:47.466+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:48.280 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:48.279+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:48.460 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:48.458+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:49.236 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:49.234+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:49.476 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:49.475+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:50.189 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:50.188+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:50.455 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:50.454+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:51.222 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:51.220+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:51.493 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:51.491+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:52.201 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:52.200+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:52.532 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:52.531+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:53.248 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:53.246+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:53.522 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:53.518+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:54.220 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:54.220+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:54.502 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:54.502+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:55.249 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:55.249+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:55.523 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:55.522+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:56.212 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:56.212+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:56.565 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:56.565+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:57.222 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:57.222+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:57.591 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:57.591+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:58.224 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:58.224+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:58.575 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:58.575+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:59.207 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:34:59.206+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:34:59.584 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:34:59.584+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:00.160 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:00.159+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:00.578 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:00.577+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:01.158 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:01.157+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:01.543 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:01.543+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:02.191 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:02.191+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:02.519 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:02.519+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:03.151 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:03.151+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:03.558 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:03.558+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:04.127 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:04.127+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:04.598 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:04.597+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:05.128 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:05.128+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:05.565 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:05.565+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:06.087 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:06.087+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:06.534 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:06.533+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:07.098 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:07.098+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:07.496 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:07.495+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:08.071 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:08.071+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:08.455 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:08.454+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:09.101 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:09.099+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:09.446 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:09.446+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:10.132 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:10.132+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:10.398 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:10.397+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:11.179 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:11.178+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:11.381 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:11.380+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:12.225 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:12.223+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:12.355 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:12.354+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:13.216 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:13.215+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:13.323 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:13.322+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:14.223 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:14.222+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:14.330 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:14.329+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:15.263 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:15.261+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:15.322 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:15.320+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:16.257 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:16.255+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:16.305 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:16.304+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:17.239 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:17.238+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:17.295 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:17.294+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:18.275 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:18.274+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:18.341 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:18.340+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:19.228 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:19.226+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:19.389 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:19.388+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:20.216 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:20.214+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:20.413 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:20.412+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:21.169 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:21.168+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:21.398 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:21.396+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:22.184 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:22.182+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:22.395 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:22.394+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:23.138 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:23.136+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:23.419 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:23.418+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:24.108 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:24.107+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:24.443 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:24.441+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:25.083 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:25.082+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:25.478 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:25.476+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:26.056 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:26.054+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:26.450 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:26.449+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:27.060 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:27.059+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:27.464 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:27.462+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:28.078 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:28.077+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:28.422 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:28.420+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:29.073 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:29.071+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:29.405 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:29.403+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:30.098 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:30.096+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:30.419 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:30.418+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:31.118 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:31.117+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:31.381 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:31.380+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:32.123 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:32.122+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:32.339 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:32.338+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:33.080 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:33.078+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:33.298 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:33.296+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:34.120 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:34.119+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:34.320 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:34.318+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:35.147 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:35.145+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:35.368 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:35.366+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:36.181 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:36.180+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:36.415 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:36.414+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:37.200 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:37.198+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:37.374 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:37.373+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:38.223 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:38.222+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:38.405 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:38.403+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:39.219 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:39.218+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:39.363 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:39.361+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:40.190 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:40.188+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:40.321 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:40.320+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:41.219 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:41.218+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:41.348 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:41.347+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:42.175 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:42.174+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:42.391 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:42.389+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:43.172 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:43.170+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:43.346 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:43.344+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:44.129 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:44.129+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:44.358 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:44.357+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:45.142 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:45.141+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:45.360 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:45.360+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:46.144 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:46.144+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:46.406 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:46.406+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:47.166 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:47.166+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:47.376 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:47.376+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:48.203 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:48.202+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:48.426 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:48.426+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:49.214 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:49.214+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:49.446 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:49.446+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:50.196 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:50.195+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:50.450 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:50.449+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:51.157 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:51.157+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:51.417 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:51.416+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:52.166 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:52.165+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:52.377 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:52.376+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:53.137 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:53.136+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:53.332 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:53.331+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:54.099 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:54.098+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:54.336 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:54.335+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:55.136 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:55.136+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:55.382 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:55.382+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:56.163 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:56.162+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:56.406 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:56.406+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:57.192 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:57.191+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:57.402 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:57.401+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:58.209 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:58.208+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:58.447 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:58.447+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:59.251 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:35:59.251+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:35:59.477 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:35:59.476+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:00.278 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:00.277+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:00.494 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:00.493+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:01.309 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:01.308+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:01.489 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:01.488+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:02.342 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:02.340+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:02.525 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:02.523+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:03.367 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:03.366+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:03.558 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:03.557+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:04.326 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:04.324+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:04.534 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:04.533+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:05.301 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:05.300+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:05.513 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:05.511+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:06.252 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:06.251+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:06.511 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:06.510+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:07.248 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:07.247+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:07.519 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:07.518+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:08.283 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:08.281+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:08.569 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:08.568+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:09.246 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:09.244+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:09.558 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:09.556+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:10.196 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:10.195+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:10.571 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:10.569+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:11.189 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:11.187+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:11.594 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:11.592+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:12.176 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:12.175+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:12.614 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:12.613+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:13.150 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:13.149+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:13.568 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:13.566+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:14.126 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:14.125+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:14.529 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:14.528+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:15.146 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:15.145+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:15.574 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:15.573+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:16.179 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:16.178+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:16.612 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:16.611+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:17.225 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:17.223+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:17.602 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:17.600+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:18.209 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:18.207+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:18.619 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:18.618+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:19.232 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:19.230+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:19.588 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:19.586+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:20.229 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:20.227+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:20.547 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:20.545+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:21.194 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:21.193+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:21.541 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:21.538+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:22.198 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:22.197+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:22.520 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:22.519+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:23.180 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:23.179+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:23.555 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:23.553+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:24.230 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:24.229+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:24.563 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:24.562+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:25.220 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:25.218+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:25.536 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:25.535+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:26.190 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:26.188+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:26.524 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:26.523+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:27.184 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:27.183+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:27.518 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:27.516+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:28.202 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:28.200+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:28.472 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:28.471+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:29.159 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:29.158+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:29.521 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:29.520+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:30.169 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:30.167+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:30.502 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:30.501+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:31.128 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:31.127+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:31.485 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:31.484+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:32.169 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:32.167+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:32.457 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:32.455+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:33.146 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:33.145+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:33.411 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:33.411+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:34.125 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:34.125+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:34.424 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:34.423+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:35.105 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:35.104+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:35.391 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:35.391+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:36.135 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:36.134+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:36.387 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:36.387+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:37.091 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:37.090+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:37.380 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:37.379+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:38.131 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:38.130+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:38.402 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:38.401+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:39.179 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:39.178+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:39.393 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:39.393+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:40.163 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:40.162+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:40.381 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:40.381+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:41.130 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:41.129+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:41.345 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:41.345+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:42.116 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:42.116+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:42.344 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:42.344+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:43.118 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:43.118+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:43.310 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:43.310+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:44.133 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:44.131+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:44.296 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:44.296+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:45.129 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:45.129+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:45.251 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:45.251+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:46.171 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:46.171+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:46.213 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:46.213+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:47.189 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:47.189+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:47.245 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:47.245+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:48.197 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:48.197+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:48.223 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:48.223+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:49.181 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:49.181+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:49.213 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:49.213+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:50.200 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:50.199+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:50.227 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:50.227+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:51.180 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:51.179+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:51.219 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:51.218+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:52.178 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:52.176+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:52.221 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:52.220+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:53.218 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:53.217+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:53.223 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:53.222+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:54.177 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:54.174+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:54.242 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:54.240+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:55.133 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:55.132+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:55.272 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:55.270+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:56.137 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:56.136+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:56.243 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:56.242+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:57.182 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:57.180+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:57.274 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:57.272+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:58.228 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:58.226+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:58.256 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:58.255+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:59.203 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:36:59.202+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:36:59.297 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:36:59.296+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:00.173 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:00.172+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:00.317 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:00.314+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:01.206 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:01.205+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:01.351 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:01.349+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:02.230 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:02.229+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:02.382 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:02.381+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:03.181 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:03.180+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:03.342 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:03.340+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:04.180 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:04.178+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:04.305 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:04.303+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:05.228 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:05.226+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:05.261 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:05.259+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:06.184 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:06.183+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:06.295 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:06.294+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:07.196 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:07.195+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:07.271 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:07.269+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:08.220 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:08.219+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:08.275 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:08.273+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:09.220 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:09.219+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:09.235 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:09.234+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:10.204 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:10.202+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:10.217 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:10.216+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:11.164 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:11.163+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:11.225 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:11.224+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:12.143 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:12.142+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:12.198 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:12.196+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:13.136 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:13.135+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:13.175 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:13.174+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:14.126 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:14.125+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:14.169 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:14.168+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:15.109 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:15.108+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:15.159 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:15.157+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:16.111 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:16.109+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:16.163 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:16.161+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:17.130 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:17.128+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:17.167 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:17.166+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:18.127 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:18.126+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:18.159 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:18.157+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:19.108 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:19.106+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:19.162 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:19.161+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:20.109 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:20.108+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:20.113 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:20.112+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:21.096 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:21.094+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:21.119 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:21.118+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:22.064 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:22.063+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:22.126 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:22.125+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:23.097 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:23.096+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:23.173 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:23.171+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:24.105 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:24.104+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:24.159 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:24.159+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:25.070 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:25.069+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:25.111 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:25.111+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:26.057 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:26.056+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:26.090 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:26.089+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:27.088 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:27.087+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:27.103 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:27.102+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:28.111 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:28.111+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:28.130 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:28.129+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:29.105 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:29.105+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:29.152 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:29.151+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:30.061 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:30.061+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:30.107 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:30.107+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:31.098 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:31.097+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:31.140 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:31.140+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:32.142 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:32.141+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:32.181 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:32.181+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:33.145 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:33.144+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:33.210 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:33.210+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:34.175 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:34.175+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:34.260 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:34.260+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:35.193 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:35.192+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:35.239 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:35.239+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:36.157 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:36.157+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:36.261 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:36.260+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:37.161 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:37.160+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:37.235 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:37.234+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:38.161 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:38.161+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:38.227 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:38.226+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:39.121 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:39.121+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:39.274 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:39.274+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:40.135 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:40.134+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:40.285 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:40.285+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:41.142 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:41.140+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:41.284 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:41.283+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:42.097 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:42.095+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:42.335 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:42.333+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:43.071 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:43.070+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:43.334 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:43.333+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:44.039 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:44.038+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:44.357 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:44.356+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:44.993 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:44.992+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:45.399 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:45.398+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:45.983 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:45.981+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:46.357 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:46.356+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:46.959 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:46.958+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:47.344 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:47.342+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:48.003 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:48.002+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:48.313 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:48.312+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:49.049 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:49.047+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:49.347 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:49.345+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:50.097 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:50.095+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:50.315 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:50.314+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:51.107 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:51.106+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:51.292 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:51.291+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:52.153 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:52.151+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:52.343 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:52.342+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:53.196 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:53.195+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:53.385 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:53.383+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:54.209 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:54.207+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:54.430 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:54.429+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:55.200 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:55.198+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:55.397 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:55.396+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:56.203 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:56.202+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:56.371 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:56.370+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:57.253 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:57.252+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:57.407 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:57.406+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:58.295 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:58.294+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:58.404 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:58.403+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:59.286 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:37:59.284+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:37:59.413 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:37:59.412+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:00.315 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:00.313+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:00.405 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:00.404+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:01.269 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:01.268+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:01.360 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:01.359+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:02.258 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:02.256+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:02.323 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:02.321+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:03.246 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:03.244+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:03.307 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:03.306+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:04.229 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:04.227+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:04.323 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:04.322+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:05.230 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:05.228+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:05.349 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:05.348+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:06.263 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:06.262+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:06.393 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:06.391+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:07.309 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:07.308+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:07.354 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:07.352+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:08.284 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:08.283+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:08.337 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:08.335+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:09.300 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:09.298+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:09.358 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:09.357+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:10.307 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:10.305+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:10.353 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:10.348+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:11.283 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:11.282+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:11.359 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:11.358+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:12.252 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:12.250+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:12.352 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:12.351+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:13.209 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:13.209+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:13.332 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:13.331+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:14.206 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:14.206+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:14.304 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:14.304+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:15.215 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:15.215+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:15.325 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:15.324+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:16.209 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:16.209+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:16.370 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:16.369+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:17.168 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:17.167+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:17.386 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:17.386+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:18.168 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:18.168+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:18.410 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:18.409+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:19.192 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:19.191+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:19.430 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:19.430+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:20.144 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:20.143+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:20.439 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:20.438+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:21.106 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:21.106+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:21.471 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:21.471+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:22.078 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:22.078+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:22.514 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:22.514+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:23.056 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:23.055+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:23.532 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:23.531+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:24.083 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:24.083+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:24.500 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:24.500+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:25.077 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:25.076+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:25.506 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:25.505+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:26.120 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:26.120+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:26.525 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:26.524+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:27.091 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:27.091+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:27.486 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:27.484+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:28.060 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:28.059+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:28.520 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:28.520+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:29.096 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:29.096+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:29.566 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:29.566+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:30.069 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:30.068+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:30.610 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:30.609+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:31.098 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:31.097+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:31.649 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:31.647+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:32.074 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:32.073+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:32.667 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:32.665+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:33.045 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:33.044+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:33.698 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:33.697+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:34.022 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:34.021+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:34.653 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:34.651+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:34.988 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:34.986+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:35.672 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:35.670+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:36.038 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:36.037+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:36.623 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:36.622+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:37.000 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:36.999+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:37.582 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:37.581+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:38.050 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:38.049+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:38.534 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:38.533+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:39.019 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:39.018+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:39.504 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:39.502+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:40.060 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:40.058+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:40.525 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:40.523+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:41.100 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:41.098+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:41.544 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:41.543+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:42.093 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:42.091+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:42.577 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:42.575+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:43.092 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:43.090+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:43.573 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:43.572+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:44.065 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:44.064+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:44.552 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:44.551+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:45.043 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:45.042+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:45.534 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:45.532+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:46.018 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:46.017+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:46.536 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:46.535+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:47.037 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:47.035+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:47.582 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:47.580+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:48.047 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:48.045+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:48.602 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:48.601+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:49.040 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:49.039+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:49.629 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:49.627+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:50.031 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:50.030+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:50.627 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:50.625+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:51.068 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:51.066+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:51.666 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:51.664+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:52.104 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:52.103+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:52.641 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:52.640+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:53.120 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:53.119+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:53.657 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:53.655+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:54.072 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:54.070+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:54.697 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:54.696+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:55.080 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:55.079+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:55.682 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:55.681+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:56.104 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:56.103+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:56.688 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:56.687+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:57.136 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:57.135+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:57.735 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:57.733+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:58.107 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:58.105+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:58.700 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:58.699+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:59.156 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:38:59.155+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:38:59.708 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:38:59.707+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:00.114 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:00.112+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:00.659 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:00.658+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:01.161 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:01.160+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:01.618 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:01.616+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:02.156 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:02.155+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:02.627 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:02.626+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:03.110 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:03.109+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:03.664 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:03.664+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:04.074 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:04.073+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:04.624 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:04.624+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:05.114 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:05.114+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:05.597 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:05.596+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:06.114 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:06.114+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:06.567 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:06.567+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:07.091 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:07.091+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:07.527 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:07.527+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:08.137 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:08.137+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:08.519 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:08.518+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:09.149 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:09.148+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:09.543 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:09.543+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:10.178 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:10.177+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:10.550 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:10.549+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:11.194 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:11.194+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:11.595 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:11.594+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:12.155 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:12.155+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:12.554 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:12.553+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:13.184 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:13.183+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:13.593 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:13.592+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:14.214 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:14.214+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:14.640 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:14.640+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:15.255 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:15.255+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:15.670 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:15.669+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:16.253 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:16.252+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:16.653 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:16.653+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:17.264 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:17.264+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:17.629 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:17.628+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:18.252 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:18.252+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:18.614 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:18.614+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:19.226 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:19.226+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:19.587 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:19.586+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:20.254 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:20.253+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:20.551 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:20.550+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:21.253 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:21.252+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:21.569 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:21.568+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:22.210 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:22.209+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:22.565 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:22.564+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:23.258 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:23.256+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:23.566 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:23.565+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:24.244 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:24.243+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:24.560 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:24.559+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:25.244 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:25.242+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:25.515 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:25.514+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:26.233 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:26.231+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:26.556 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:26.554+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:27.258 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:27.256+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:27.517 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:27.515+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:28.258 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:28.257+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:28.551 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:28.550+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:29.289 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:29.287+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:29.542 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:29.540+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:30.270 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:30.269+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:30.494 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:30.493+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:31.299 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:31.298+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:31.491 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:31.489+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:32.291 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:32.290+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:32.499 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:32.497+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:33.257 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:33.256+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:33.491 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:33.490+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:34.252 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:34.250+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:34.471 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:34.470+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:35.302 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:35.300+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:35.493 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:35.492+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:36.342 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:36.340+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:36.506 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:36.504+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:37.353 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:37.352+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:37.526 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:37.525+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:38.334 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:38.333+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:38.534 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:38.533+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:39.325 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:39.323+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:39.494 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:39.492+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:40.351 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:40.349+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:40.535 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:40.533+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:41.313 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:41.311+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:41.532 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:41.530+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:42.281 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:42.279+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:42.568 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:42.567+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:43.274 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:43.273+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:43.598 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:43.595+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:44.252 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:44.251+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:44.555 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:44.554+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:45.275 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:45.273+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:45.570 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:45.568+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:46.248 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:46.247+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:46.613 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:46.611+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:47.253 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:47.251+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:47.661 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:47.660+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:48.232 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:48.230+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:48.652 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:48.651+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:49.266 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:49.264+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:49.631 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:49.630+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:50.234 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:50.233+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:50.613 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:50.612+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:51.212 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:51.211+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:51.623 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:51.621+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:52.216 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:52.215+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:52.582 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:52.581+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:53.207 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:53.206+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:53.598 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:53.598+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:54.223 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:54.222+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:54.628 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:54.628+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:55.263 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:55.262+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:55.677 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:55.676+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:56.263 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:56.263+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:56.649 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:56.648+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:57.256 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:57.256+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:57.678 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:57.678+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:58.299 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:58.298+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:58.635 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:58.634+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:59.258 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:39:59.258+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:39:59.666 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:39:59.665+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:00.251 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:00.250+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:00.698 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:00.697+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:01.250 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:01.250+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:01.661 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:01.661+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:02.217 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:02.217+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:02.655 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:02.654+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:03.176 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:03.176+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:03.665 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:03.665+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:04.150 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:04.149+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:04.666 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:04.666+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:05.145 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:05.144+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:05.685 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:05.684+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:06.099 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:06.099+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:06.715 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:06.715+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:07.086 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:07.085+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:07.728 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:07.728+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:08.076 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:08.076+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:08.732 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:08.732+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:09.071 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:09.069+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:09.710 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:09.710+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:10.034 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:10.032+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:10.680 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:10.678+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:11.073 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:11.072+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:11.729 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:11.728+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:12.123 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:12.121+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:12.756 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:12.754+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:13.142 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:13.140+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:13.717 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:13.715+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:14.170 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:14.169+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:14.706 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:14.705+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:15.129 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:15.127+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:15.668 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:15.666+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:16.086 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:16.084+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:16.657 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:16.655+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:17.051 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:17.049+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:17.702 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:17.701+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:18.055 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:18.053+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:18.716 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:18.714+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:19.034 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:19.032+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:19.704 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:19.702+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:20.045 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:20.043+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:20.688 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:20.687+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:21.053 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:21.051+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:21.663 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:21.661+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:22.077 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:22.075+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:22.661 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:22.660+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:23.061 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:23.059+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:23.667 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:23.666+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:24.077 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:24.075+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:24.643 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:24.641+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:25.101 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:25.100+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:25.617 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:25.615+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:26.136 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:26.135+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:26.639 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:26.637+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:27.148 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:27.146+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:27.637 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:27.636+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:28.160 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:28.159+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:28.654 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:28.653+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:29.133 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:29.132+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:29.610 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:29.609+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:30.165 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:30.164+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:30.595 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:30.593+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:31.202 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:31.201+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:31.638 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:31.636+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:32.162 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:32.160+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:32.609 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:32.607+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:33.203 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:33.201+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:33.635 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:33.634+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:34.246 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:34.245+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:34.589 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:34.588+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:35.263 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:35.262+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:35.637 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:35.636+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:36.282 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:36.279+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:36.606 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:36.605+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:37.272 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:37.271+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:37.577 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:37.575+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:38.244 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:38.242+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:38.579 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:38.577+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:39.248 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:39.247+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:39.602 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:39.600+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:40.265 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:40.263+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:40.608 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:40.607+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:41.238 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:41.238+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:41.582 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:41.581+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:42.257 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:42.256+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:42.616 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:42.616+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:43.250 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:43.249+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:43.620 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:43.619+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:44.227 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:44.227+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:44.654 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:44.653+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:45.221 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:45.220+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:45.616 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:45.616+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:46.219 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:46.219+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:46.570 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:46.569+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:47.211 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:47.210+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:47.610 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:47.609+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:48.199 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:48.198+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:48.565 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:48.564+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:49.238 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:49.238+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:49.519 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:49.518+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:50.206 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:50.206+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:50.472 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:50.472+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:51.253 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:51.252+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:51.466 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:51.462+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:52.249 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:52.248+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:52.471 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:52.471+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:53.252 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:53.251+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:53.467 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:53.466+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:54.262 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:54.261+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:54.434 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:54.434+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:55.242 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:55.241+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:55.479 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:55.478+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:56.222 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:56.221+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:56.434 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:56.434+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:57.257 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:57.257+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:57.388 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:57.388+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:58.294 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:58.293+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:58.416 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:58.415+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:59.259 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:40:59.258+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:40:59.375 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:40:59.375+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:00.246 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:00.245+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:00.407 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:00.405+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:01.254 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:01.253+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:01.401 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:01.399+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:02.225 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:02.223+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:02.362 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:02.361+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:03.219 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:03.218+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:03.361 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:03.360+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:04.197 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:04.193+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:04.375 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:04.373+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:05.210 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:05.209+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:05.422 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:05.421+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:06.184 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:06.182+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:06.420 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:06.419+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:07.174 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:07.173+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:07.446 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:07.444+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:08.126 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:08.124+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:08.446 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:08.445+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:09.143 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:09.141+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:09.439 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:09.438+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:10.100 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:10.098+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:10.471 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:10.469+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:11.092 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:11.090+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:11.423 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:11.422+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:12.094 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:12.091+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:12.381 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:12.380+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:13.073 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:13.072+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:13.356 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:13.354+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:14.059 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:14.058+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:14.320 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:14.319+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:15.047 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:15.046+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:15.309 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:15.307+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:16.028 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:16.027+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:16.327 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:16.325+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:17.016 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:17.015+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:17.304 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:17.303+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:18.023 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:18.022+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:18.279 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:18.278+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:19.006 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:19.005+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:19.325 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:19.324+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:20.047 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:20.045+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:20.330 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:20.329+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:21.034 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:21.032+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:21.331 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:21.330+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:21.990 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:21.988+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:22.366 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:22.365+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:23.024 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:23.022+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:23.356 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:23.355+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:24.064 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:24.062+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:24.310 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:24.308+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:25.067 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:25.066+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:25.300 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:25.299+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:26.113 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:26.111+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:26.304 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:26.303+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:27.154 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:27.152+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:27.345 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:27.343+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:28.108 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:28.106+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:28.301 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:28.299+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:29.141 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:29.140+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:29.324 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:29.323+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:30.189 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:30.188+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:30.365 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:30.363+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:31.146 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:31.145+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:31.350 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:31.348+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:32.167 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:32.166+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:32.321 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:32.320+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:33.216 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:33.216+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:33.354 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:33.336+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:34.230 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:34.230+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:34.331 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:34.330+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:35.262 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:35.262+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:35.360 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:35.359+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:36.218 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:36.214+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:36.375 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:36.375+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:37.205 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:37.204+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:37.355 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:37.355+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:38.223 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:38.222+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:38.335 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:38.334+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:39.238 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:39.238+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:39.300 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:39.299+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:40.272 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:40.271+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:40.326 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:40.326+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:41.226 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:41.225+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:41.330 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:41.330+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:42.226 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:42.225+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:42.299 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:42.299+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:43.228 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:43.227+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:43.324 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:43.323+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:44.233 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:44.232+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:44.302 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:44.301+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:45.233 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:45.232+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:45.319 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:45.318+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:46.231 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:46.230+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:46.290 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:46.290+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:47.213 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:47.213+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:47.274 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:47.273+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:48.221 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:48.220+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:48.226 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:48.225+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:49.251 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:49.249+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:49.265 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:49.263+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:50.256 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:50.254+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:50.266 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:50.265+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:51.218 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:51.216+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:51.245 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:51.244+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:52.212 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:52.211+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:52.230 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:52.228+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:53.229 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:53.227+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:53.257 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:53.255+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:54.218 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:54.217+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:54.241 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:54.239+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:55.193 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:55.192+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:55.244 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:55.242+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:56.228 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:56.226+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:56.290 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:56.288+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:57.276 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:57.274+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:57.339 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:57.337+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:58.294 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:58.293+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:58.323 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:58.322+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:59.282 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:41:59.280+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:41:59.334 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:41:59.333+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:00.259 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:00.258+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:00.306 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:00.304+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:01.261 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:01.260+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:01.270 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:01.269+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:02.239 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:02.237+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:02.284 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:02.283+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:03.243 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:03.241+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:03.281 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:03.280+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:04.223 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:04.222+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:04.245 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:04.243+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:05.213 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:05.211+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:05.230 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:05.229+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:06.241 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:06.239+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:06.249 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:06.248+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:07.221 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:07.219+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:07.228 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:07.226+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:08.239 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:08.238+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:08.254 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:08.253+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:09.229 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:09.228+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:09.235 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:09.234+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:10.189 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:10.187+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:10.215 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:10.214+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:11.163 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:11.161+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:11.220 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:11.218+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:12.174 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:12.173+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:12.195 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:12.193+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:13.177 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:13.176+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:13.244 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:13.243+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:14.190 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:14.188+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:14.225 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:14.223+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:15.177 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:15.175+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:15.225 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:15.224+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:16.136 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:16.135+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:16.250 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:16.248+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:17.147 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:17.145+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:17.205 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:17.203+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:18.170 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:18.168+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:18.212 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:18.211+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:19.133 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:19.131+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:19.214 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:19.212+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:20.119 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:20.117+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:20.167 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:20.166+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:21.133 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:21.132+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:21.203 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:21.202+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:22.130 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:22.130+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:22.218 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:22.218+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:23.143 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:23.143+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:23.256 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:23.255+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:24.126 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:24.126+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:24.274 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:24.274+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:25.166 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:25.166+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:25.234 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:25.233+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:26.155 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:26.154+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:26.241 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:26.241+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:27.199 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:27.199+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:27.246 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:27.246+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:28.186 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:28.186+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:28.286 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:28.285+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:29.178 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:29.178+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:29.286 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:29.285+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:30.182 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:30.181+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:30.258 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:30.258+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:31.208 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:31.207+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:31.252 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:31.251+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:32.251 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:32.250+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:32.282 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:32.282+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:33.220 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:33.219+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:33.303 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:33.303+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:34.211 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:34.211+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:34.301 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:34.301+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:35.216 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:35.215+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:35.345 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:35.345+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:36.257 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:36.257+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:36.380 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:36.379+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:37.222 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:37.221+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:37.391 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:37.391+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:38.191 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:38.191+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:38.390 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:38.389+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:39.239 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:39.237+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:39.362 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:39.360+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:40.212 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:40.210+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:40.334 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:40.333+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:41.253 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:41.252+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:41.325 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:41.323+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:42.216 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:42.214+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:42.321 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:42.320+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:43.257 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:43.255+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:43.287 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:43.286+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:44.254 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:44.252+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:44.337 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:44.336+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:45.293 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:45.292+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:45.386 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:45.385+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:46.281 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:46.279+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:46.340 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:46.339+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:47.239 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:47.237+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:47.314 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:47.313+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:48.241 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:48.239+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:48.348 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:48.347+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:49.247 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:49.245+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:49.384 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:49.383+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:50.269 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:50.267+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:50.379 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:50.378+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:51.295 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:51.293+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:51.418 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:51.416+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:52.345 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:52.344+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:52.439 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:52.437+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:53.387 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:53.386+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:53.392 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:53.391+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:54.425 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:54.423+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:54.426 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:54.424+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:55.401 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:55.400+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:55.447 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:55.445+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:56.428 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:56.426+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:56.433 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:56.431+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:57.398 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:57.396+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:57.416 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:57.415+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:58.441 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:58.439+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:58.461 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:58.459+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:59.442 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:42:59.440+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:42:59.455 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:42:59.454+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:00.449 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:00.447+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:00.463 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:00.462+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:01.437 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:01.436+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:01.481 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:01.480+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:02.435 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:02.434+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:02.483 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:02.482+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:03.446 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:03.445+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:03.493 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:03.491+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:04.427 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:04.425+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:04.460 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:04.459+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:05.412 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:05.411+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:05.464 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:05.463+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:06.382 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:06.381+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:06.450 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:06.448+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:07.404 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:07.403+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:07.488 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:07.487+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:08.377 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:08.375+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:08.514 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:08.513+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:09.425 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:09.424+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:09.471 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:09.469+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:10.430 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:10.428+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:10.477 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:10.476+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:11.447 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:11.447+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:11.489 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:11.489+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:12.431 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:12.430+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:12.486 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:12.486+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:13.408 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:13.408+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:13.448 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:13.447+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:14.423 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:14.422+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:14.456 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:14.456+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:15.455 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:15.454+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:15.481 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:15.481+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:16.477 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:16.477+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:16.507 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:16.506+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:17.470 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:17.469+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:17.524 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:17.523+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:18.446 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:18.445+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:18.529 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:18.529+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:19.463 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:19.462+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:19.510 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:19.510+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:20.485 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:20.483+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:20.499 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:20.498+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:21.519 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:21.518+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:21.549 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:21.548+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:22.521 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:22.520+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:22.541 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:22.539+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:23.539 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:23.538+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:23.558 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:23.558+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:24.536 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:24.535+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:24.537 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:24.537+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:25.510 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:25.509+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:25.576 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:25.575+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:26.559 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:26.559+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:26.560 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:26.560+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:27.581 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:27.581+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:27.603 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:27.603+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:28.609 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:28.607+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:28.638 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:28.637+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:29.632 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:29.630+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:29.645 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:29.644+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:30.592 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:30.591+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:30.657 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:30.655+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:31.627 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:31.625+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:31.681 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:31.679+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:32.599 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:32.598+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:32.727 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:32.726+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:33.611 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:33.609+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:33.730 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:33.729+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:34.594 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:34.593+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:34.755 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:34.754+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:35.637 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:35.635+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:35.712 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:35.711+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:36.679 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:36.678+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:36.738 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:36.736+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:37.709 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:37.707+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:37.755 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:37.753+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:38.759 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:38.757+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:38.759 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:38.757+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:39.771 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:39.769+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:39.772 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:39.771+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:40.753 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:40.752+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:40.769 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:40.768+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:41.739 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:41.737+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:41.793 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:41.792+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:42.785 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:42.784+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:42.790 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:42.788+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:43.818 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:43.816+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:43.827 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:43.826+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:44.812 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:44.811+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:44.848 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:44.847+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:45.791 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:45.789+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:45.880 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:45.878+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:46.777 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:46.775+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:46.842 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:46.841+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:47.806 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:47.804+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:47.846 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:47.844+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:48.851 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:48.849+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:48.872 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:48.870+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:49.823 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:49.821+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:49.851 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:49.850+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:50.831 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:50.830+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:50.856 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:50.855+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:51.839 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:51.837+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:51.862 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:51.860+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:52.826 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:52.824+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:52.831 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:52.829+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:53.829 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:53.827+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:53.835 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:53.833+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:54.830 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:54.828+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:54.845 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:54.843+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:55.831 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:55.829+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:55.836 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:55.834+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:56.804 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:56.802+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:56.865 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:56.864+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:57.831 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:57.829+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:57.850 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:57.849+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:58.851 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:58.849+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:58.899 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:58.898+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:59.833 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:43:59.831+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:43:59.875 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:43:59.874+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:00.845 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:00.843+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:00.883 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:00.882+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:01.821 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:01.821+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:01.922 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:01.921+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:02.806 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:02.805+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:02.902 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:02.901+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:03.804 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:03.803+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:03.863 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:03.863+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:04.812 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:04.811+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:04.873 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:04.872+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:05.836 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:05.835+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:05.851 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:05.851+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:06.873 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:06.872+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:06.879 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:06.879+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:07.879 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:07.878+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:07.904 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:07.904+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:08.899 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:08.899+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:08.907 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:08.907+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:09.856 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:09.856+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:09.940 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:09.940+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:10.854 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:10.854+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:10.983 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:10.983+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:11.852 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:11.851+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:11.957 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:11.957+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:12.809 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:12.808+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:13.005 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:13.004+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:13.794 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:13.794+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:14.052 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:14.051+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:14.814 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:14.814+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:15.059 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:15.058+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:15.785 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:15.784+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:16.033 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:16.032+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:16.800 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:16.800+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:17.031 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:17.030+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:17.776 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:17.775+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:18.023 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:18.022+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:18.809 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:18.807+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:19.056 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:19.055+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:19.839 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:19.837+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:20.027 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:20.025+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:20.814 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:20.812+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:21.041 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:21.040+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:21.817 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:21.815+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:22.002 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:22.000+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:22.851 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:22.850+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:23.004 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:23.003+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:23.847 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:23.845+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:24.041 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:24.039+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:24.799 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:24.797+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:25.034 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:25.032+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:25.804 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:25.803+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:25.987 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:25.985+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:26.811 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:26.809+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:26.992 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:26.991+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:27.831 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:27.830+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:27.982 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:27.980+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:28.823 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:28.821+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:28.988 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:28.987+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:29.793 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:29.791+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:29.952 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:29.951+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:30.812 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:30.810+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:30.967 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:30.965+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:31.763 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:31.761+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:31.927 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:31.926+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:32.745 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:32.743+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:32.949 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:32.947+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:33.737 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:33.735+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:33.949 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:33.947+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:34.755 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:34.753+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:34.955 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:34.953+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:35.720 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:35.718+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:35.998 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:35.996+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:36.693 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:36.691+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:36.986 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:36.985+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:37.691 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:37.689+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:38.030 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:38.028+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:38.731 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:38.729+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:39.002 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:39.000+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:39.693 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:39.692+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:40.042 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:40.040+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:40.647 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:40.645+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:41.017 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:41.015+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:41.610 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:41.609+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:42.019 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:42.018+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:42.606 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:42.604+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:42.978 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:42.976+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:43.619 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:43.617+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:43.990 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:43.989+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:44.663 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:44.661+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:44.983 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:44.982+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:45.700 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:45.698+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:45.962 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:45.961+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:46.721 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:46.719+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:46.921 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:46.919+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:47.769 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:47.767+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:47.934 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:47.932+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:48.746 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:48.745+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:48.939 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:48.937+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:49.722 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:49.721+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:49.891 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:49.889+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:50.701 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:50.699+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:50.925 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:50.925+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:51.744 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:51.743+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:51.926 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:51.925+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:52.726 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:52.725+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:52.970 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:52.969+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:53.760 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:53.759+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:53.923 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:53.923+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:54.746 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:54.745+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:54.882 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:54.881+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:55.722 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:55.721+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:55.845 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:55.844+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:56.707 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:56.706+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:56.803 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:56.802+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:57.678 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:57.677+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:57.788 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:57.787+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:58.684 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:58.684+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:58.760 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:58.759+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:59.656 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:44:59.655+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:44:59.806 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:44:59.805+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:00.610 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:00.609+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:00.835 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:00.834+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:01.630 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:01.629+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:01.788 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:01.787+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:02.596 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:02.595+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:02.806 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:02.806+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:03.593 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:03.592+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:03.830 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:03.830+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:04.624 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:04.623+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:04.864 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:04.863+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:05.620 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:05.619+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:05.889 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:05.888+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:06.667 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:06.666+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:06.864 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:06.863+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:07.660 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:07.659+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:07.893 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:07.892+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:08.623 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:08.621+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:08.925 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:08.923+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:09.589 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:09.588+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:09.975 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:09.973+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:10.579 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:10.577+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:10.970 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:10.968+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:11.563 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:11.561+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:12.008 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:12.006+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:12.589 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:12.587+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:13.054 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:13.053+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:13.550 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:13.548+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:14.035 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:14.033+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:14.521 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:14.519+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:15.008 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:15.006+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:15.559 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:15.557+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:15.979 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:15.977+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:16.593 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:16.591+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:16.981 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:16.980+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:17.637 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:17.636+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:17.980 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:17.978+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:18.610 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:18.609+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:19.022 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:19.020+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:19.566 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:19.564+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:20.032 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:20.030+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:20.563 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:20.561+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:20.991 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:20.989+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:21.525 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:21.523+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:22.038 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:22.037+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:22.478 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:22.476+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:23.032 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:23.030+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:23.517 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:23.515+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:23.993 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:23.991+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:24.526 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:24.524+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:24.953 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:24.951+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:25.484 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:25.482+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:26.000 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:25.998+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:26.452 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:26.451+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:26.983 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:26.982+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:27.445 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:27.444+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:27.998 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:27.996+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:28.422 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:28.420+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:28.997 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:28.995+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:29.378 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:29.376+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:29.984 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:29.982+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:30.329 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:30.327+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:31.000 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:30.998+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:31.348 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:31.346+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:32.049 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:32.047+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:32.341 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:32.339+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:33.047 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:33.045+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:33.361 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:33.359+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:34.091 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:34.089+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:34.360 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:34.359+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:35.083 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:35.081+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:35.410 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:35.408+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:36.062 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:36.060+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:36.412 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:36.410+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:37.106 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:37.104+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:37.392 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:37.390+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:38.124 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:38.122+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:38.432 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:38.430+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:39.149 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:39.147+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:39.443 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:39.441+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:40.161 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:40.159+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:40.439 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:40.437+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:41.136 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:41.135+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:41.482 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:41.481+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:42.144 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:42.143+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:42.449 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:42.448+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:43.155 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:43.154+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:43.407 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:43.406+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:44.134 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:44.134+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:44.387 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:44.386+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:45.088 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:45.087+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:45.421 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:45.420+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:46.106 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:46.105+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:46.449 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:46.448+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:47.118 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:47.117+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:47.466 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:47.466+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:48.141 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:48.140+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:48.465 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:48.464+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:49.103 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:49.103+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:49.454 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:49.453+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:50.148 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:50.148+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:50.479 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:50.478+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:51.110 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:51.110+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:51.433 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:51.432+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:52.119 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:52.118+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:52.450 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:52.449+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:53.085 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:53.084+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:53.448 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:53.447+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:54.055 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:54.054+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:54.427 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:54.426+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:55.067 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:55.066+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:55.405 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:55.404+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:56.099 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:56.098+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:56.441 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:56.440+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:57.097 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:57.096+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:57.426 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:57.425+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:58.133 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:58.133+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:58.387 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:58.385+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:59.143 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:45:59.141+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:45:59.347 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:45:59.345+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:00.111 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:00.109+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:00.369 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:00.367+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:01.134 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:01.132+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:01.399 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:01.397+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:02.131 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:02.129+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:02.365 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:02.363+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:03.105 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:03.103+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:03.398 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:03.396+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:04.117 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:04.115+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:04.433 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:04.431+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:05.069 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:05.067+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:05.442 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:05.440+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:06.049 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:06.047+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:06.472 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:06.470+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:07.097 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:07.095+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:07.444 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:07.442+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:08.097 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:08.095+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:08.424 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:08.423+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:09.103 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:09.101+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:09.454 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:09.452+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:10.069 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:10.067+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:10.460 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:10.458+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:11.071 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:11.069+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:11.498 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:11.496+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:12.024 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:12.022+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:12.453 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:12.451+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:13.022 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:13.020+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:13.478 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:13.476+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:13.992 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:13.991+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:14.468 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:14.466+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:14.982 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:14.980+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:15.453 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:15.449+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:15.935 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:15.933+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:16.483 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:16.481+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:16.957 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:16.955+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:17.504 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:17.503+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:17.998 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:17.996+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:18.484 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:18.482+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:18.999 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:18.997+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:19.502 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:19.500+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:20.049 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:20.047+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:20.528 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:20.526+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:21.039 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:21.038+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:21.492 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:21.490+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:22.048 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:22.046+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:22.492 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:22.490+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:23.090 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:23.088+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:23.486 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:23.485+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:24.087 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:24.085+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:24.530 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:24.528+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:25.045 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:25.043+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:25.498 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:25.496+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:26.043 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:26.041+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:26.547 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:26.545+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:27.082 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:27.080+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:27.525 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:27.523+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:28.131 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:28.129+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:28.537 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:28.535+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:29.157 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:29.155+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:29.539 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:29.537+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:30.150 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:30.148+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:30.495 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:30.493+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:31.165 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:31.163+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:31.469 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:31.468+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:32.144 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:32.143+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:32.517 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:32.516+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:33.177 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:33.177+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:33.541 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:33.540+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:34.175 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:34.174+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:34.505 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:34.504+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:35.221 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:35.220+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:35.505 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:35.504+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:36.256 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:36.255+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:36.498 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:36.497+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:37.280 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:37.279+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:37.459 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:37.458+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:38.305 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:38.304+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:38.471 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:38.470+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:39.327 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:39.326+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:39.503 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:39.502+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:40.352 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:40.351+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:40.526 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:40.525+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:41.382 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:41.381+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:41.572 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:41.571+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:42.409 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:42.408+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:42.551 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:42.550+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:43.449 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:43.448+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:43.524 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:43.523+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:44.427 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:44.426+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:44.551 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:44.550+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:45.398 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:45.397+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:45.506 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:45.505+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:46.444 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:46.443+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:46.502 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:46.501+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:47.419 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:47.418+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:47.497 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:47.492+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:48.398 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:48.397+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:48.529 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:48.528+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:49.383 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:49.381+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:49.515 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:49.513+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:50.392 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:50.390+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:50.510 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:50.508+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:51.427 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:51.425+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:51.529 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:51.528+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:52.438 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:52.437+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:52.552 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:52.550+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:53.443 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:53.441+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:53.564 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:53.562+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:54.475 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:54.473+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:54.605 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:54.603+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:55.443 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:55.441+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:55.570 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:55.568+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:56.437 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:56.435+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:56.597 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:56.595+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:57.409 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:57.407+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:57.642 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:57.640+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:58.376 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:58.374+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:58.619 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:58.617+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:59.364 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:46:59.362+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:46:59.603 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:46:59.601+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:00.365 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:00.363+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:00.633 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:00.630+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:01.322 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:01.319+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:01.627 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:01.625+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:02.313 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:02.311+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:02.620 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:02.618+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:03.346 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:03.344+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:03.596 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:03.594+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:04.306 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:04.304+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:04.615 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:04.613+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:05.297 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:05.295+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:05.627 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:05.625+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:06.333 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:06.331+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:06.624 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:06.622+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:07.374 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:07.372+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:07.622 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:07.620+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:08.419 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:08.417+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:08.635 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:08.633+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:09.422 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:09.420+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:09.609 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:09.607+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:10.448 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:10.446+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:10.561 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:10.559+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:11.446 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:11.444+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:11.537 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:11.535+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:12.490 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:12.488+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:12.517 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:12.515+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:13.484 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:13.482+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:13.530 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:13.528+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:14.437 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:14.435+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:14.547 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:14.545+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:15.414 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:15.412+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:15.595 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:15.593+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:16.369 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:16.365+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:16.643 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:16.641+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:17.325 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:17.323+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:17.674 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:17.672+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:18.294 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:18.292+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:18.663 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:18.661+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:19.334 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:19.332+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:19.675 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:19.673+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:20.290 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:20.288+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:20.659 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:20.657+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:21.306 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:21.304+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:21.700 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:21.699+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:22.289 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:22.288+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:22.735 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:22.734+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:23.319 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:23.318+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:23.736 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:23.735+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:24.361 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:24.358+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:24.716 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:24.715+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:25.393 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:25.392+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:25.684 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:25.683+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:26.357 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:26.356+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:26.636 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:26.635+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:27.377 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:27.376+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:27.643 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:27.642+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:28.357 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:28.356+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:28.607 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:28.606+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:29.373 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:29.372+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:29.597 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:29.596+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:30.356 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:30.355+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:30.552 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:30.551+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:31.344 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:31.343+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:31.564 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:31.563+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:32.366 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:32.365+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:32.595 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:32.594+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:33.408 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:33.407+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:33.636 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:33.635+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:34.360 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:34.359+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:34.638 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:34.637+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:35.380 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:35.379+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:35.674 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:35.673+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:36.410 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:36.409+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:36.708 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:36.706+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:37.456 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:37.455+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:37.682 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:37.681+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:38.482 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:38.481+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:38.699 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:38.698+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:39.511 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:39.509+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:39.724 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:39.722+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:40.480 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:40.478+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:40.752 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:40.749+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:41.467 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:41.465+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:41.744 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:41.742+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:42.439 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:42.437+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:42.713 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:42.711+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:43.403 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:43.401+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:43.666 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:43.664+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:44.420 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:44.418+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:44.647 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:44.645+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:45.448 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:45.446+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:45.691 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:45.689+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:46.489 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:46.487+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:46.731 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:46.729+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:47.473 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:47.471+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:47.685 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:47.683+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:48.427 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:48.425+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:48.730 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:48.728+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:49.433 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:49.431+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:49.732 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:49.730+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:50.403 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:50.401+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:50.730 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:50.728+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:51.360 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:51.358+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:51.760 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:51.758+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:52.325 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:52.323+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:52.767 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:52.765+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:53.338 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:53.336+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:53.757 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:53.755+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:54.382 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:54.380+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:54.724 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:54.722+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:55.432 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:55.430+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:55.696 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:55.694+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:56.419 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:56.417+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:56.687 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:56.685+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:57.399 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:57.397+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:57.699 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:57.697+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:58.409 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:58.407+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:58.702 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:58.700+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:59.456 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:47:59.454+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:47:59.718 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:47:59.716+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:00.490 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:00.488+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:00.694 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:00.692+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:01.517 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:01.515+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:01.652 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:01.650+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:02.513 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:02.511+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:02.643 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:02.641+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:03.560 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:03.558+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:03.643 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:03.641+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:04.572 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:04.570+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:04.685 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:04.683+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:05.606 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:05.604+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:05.680 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:05.678+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:06.621 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:06.619+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:06.641 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:06.639+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:07.595 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:07.593+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:07.611 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:07.609+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:08.581 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:08.579+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:08.614 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:08.612+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:09.613 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:09.611+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:09.629 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:09.627+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:10.578 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:10.576+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:10.669 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:10.667+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:11.540 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:11.537+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:11.688 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:11.686+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:12.539 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:12.538+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:12.728 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:12.727+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:13.578 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:13.577+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:13.762 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:13.760+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:14.614 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:14.613+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:14.789 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:14.788+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:15.624 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:15.622+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:15.833 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:15.832+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:16.634 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:16.633+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:16.808 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:16.806+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:17.673 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:17.672+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:17.809 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:17.808+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:18.641 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:18.640+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:18.835 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:18.834+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:19.617 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:19.616+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:19.788 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:19.787+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:20.603 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:20.602+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:20.807 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:20.806+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:21.563 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:21.562+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:21.834 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:21.833+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:22.561 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:22.560+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:22.871 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:22.870+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:23.598 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:23.597+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:23.832 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:23.831+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:24.553 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:24.552+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:24.836 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:24.835+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:25.537 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:25.535+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:25.817 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:25.816+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:26.531 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:26.529+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:26.813 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:26.812+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:27.485 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:27.484+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:27.844 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:27.843+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:28.503 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:28.502+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:28.860 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:28.859+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:29.525 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:29.523+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:29.902 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:29.900+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:30.535 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:30.533+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:30.923 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:30.921+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:31.545 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:31.543+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:31.941 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:31.939+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:32.571 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:32.569+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:32.917 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:32.915+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:33.530 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:33.528+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:33.949 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:33.947+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:34.500 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:34.497+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:34.999 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:34.997+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:35.475 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:35.473+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:35.991 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:35.989+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:36.433 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:36.431+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:36.951 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:36.949+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:37.423 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:37.421+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:37.920 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:37.918+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:38.451 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:38.449+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:38.946 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:38.944+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:39.415 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:39.413+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:39.983 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:39.981+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:40.429 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:40.427+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:40.971 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:40.969+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:41.397 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:41.394+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:41.977 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:41.975+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:42.361 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:42.359+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:42.988 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:42.986+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:43.344 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:43.342+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:44.038 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:44.036+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:44.384 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:44.382+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:45.016 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:45.014+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:45.349 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:45.347+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:46.054 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:46.052+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:46.354 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:46.352+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:47.065 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:47.063+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:47.398 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:47.396+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:48.054 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:48.052+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:48.448 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:48.446+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:49.028 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:49.026+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:49.481 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:49.479+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:49.989 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:49.987+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:50.513 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:50.511+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:51.020 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:51.018+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:51.515 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:51.513+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:51.990 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:51.988+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:52.525 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:52.523+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:53.015 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:53.012+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:53.576 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:53.574+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:54.005 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:54.003+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:54.546 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:54.544+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:55.003 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:55.001+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:55.566 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:55.564+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:56.043 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:56.041+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:56.598 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:56.596+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:57.072 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:57.069+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:57.592 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:57.590+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:58.028 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:58.026+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:58.549 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:58.547+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:59.060 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:48:59.058+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:48:59.544 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:48:59.541+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:00.057 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:00.056+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:00.541 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:00.539+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:01.091 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:01.089+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:01.513 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:01.511+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:02.110 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:02.108+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:02.486 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:02.484+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:03.105 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:03.103+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:03.515 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:03.514+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:04.139 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:04.138+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:04.565 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:04.564+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:05.180 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:05.179+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:05.539 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:05.538+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:06.143 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:06.142+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:06.548 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:06.547+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:07.113 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:07.112+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:07.508 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:07.507+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:08.066 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:08.065+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:08.507 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:08.506+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:09.030 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:09.029+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:09.513 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:09.512+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:09.988 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:09.987+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:10.512 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:10.510+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:10.990 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:10.989+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:11.538 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:11.536+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:11.988 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:11.987+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:12.542 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:12.541+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:12.970 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:12.969+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:13.513 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:13.512+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:13.949 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:13.948+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:14.523 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:14.522+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:14.960 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:14.959+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:15.509 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:15.507+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:15.956 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:15.955+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:16.538 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:16.537+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:16.936 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:16.934+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:17.521 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:17.520+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:17.930 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:17.928+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:18.550 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:18.549+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:18.906 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:18.905+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:19.573 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:19.572+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:19.897 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:19.896+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:20.574 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:20.572+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:20.915 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:20.912+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:21.613 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:21.610+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:21.929 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:21.927+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:22.575 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:22.572+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:22.904 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:22.902+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:23.580 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:23.578+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:23.922 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:23.920+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:24.614 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:24.612+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:24.971 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:24.968+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:25.641 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:25.639+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:25.928 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:25.926+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:26.611 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:26.609+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:26.888 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:26.885+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:27.597 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:27.595+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:27.846 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:27.844+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:28.572 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:28.570+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:28.802 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:28.800+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:29.597 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:29.595+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:29.792 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:29.790+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:30.595 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:30.593+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:30.794 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:30.792+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:31.632 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:31.629+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:31.785 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:31.783+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:32.662 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:32.660+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:32.818 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:32.816+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:33.664 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:33.662+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:33.822 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:33.820+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:34.694 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:34.692+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:34.810 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:34.808+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:35.724 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:35.722+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:35.859 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:35.857+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:36.730 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:36.727+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:36.836 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:36.834+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:37.771 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:37.769+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:37.884 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:37.882+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:38.800 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:38.798+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:38.917 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:38.915+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:39.793 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:39.791+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:39.938 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:39.936+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:40.794 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:40.792+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:40.911 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:40.908+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:41.794 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:41.792+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:41.878 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:41.876+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:42.811 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:42.809+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:42.883 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:42.881+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:43.803 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:43.801+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:43.906 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:43.904+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:44.805 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:44.803+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:44.907 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:44.905+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:45.787 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:45.785+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:45.911 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:45.909+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:46.772 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:46.770+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:46.885 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:46.883+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:47.797 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:47.794+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:47.905 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:47.903+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:48.764 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:48.761+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:48.890 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:48.888+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:49.814 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:49.812+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:49.861 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:49.859+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:50.827 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:50.825+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:50.857 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:50.855+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:51.801 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:51.799+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:51.847 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:51.845+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:52.827 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:52.825+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:52.829 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:52.827+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:53.782 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:53.781+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:53.801 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:53.799+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:54.779 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:54.778+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:54.826 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:54.825+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:55.808 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:55.806+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:55.866 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:55.865+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:56.801 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:56.800+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:56.872 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:56.871+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:57.822 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:57.820+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:57.859 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:57.858+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:58.815 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:58.814+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:58.858 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:58.857+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:59.827 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:49:59.826+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:49:59.904 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:49:59.902+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:00.866 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:00.864+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:00.885 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:00.884+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:01.841 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:01.840+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:01.858 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:01.857+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:02.824 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:02.823+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:02.857 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:02.856+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:03.822 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:03.820+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:03.890 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:03.889+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:04.863 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:04.862+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:04.866 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:04.865+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:05.853 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:05.852+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:05.901 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:05.900+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:06.806 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:06.805+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:06.862 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:06.861+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:07.809 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:07.808+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:07.864 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:07.863+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:08.768 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:08.766+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:08.831 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:08.830+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:09.786 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:09.785+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:09.868 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:09.867+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:10.740 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:10.738+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:10.868 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:10.866+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:11.770 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:11.768+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:11.866 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:11.864+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:12.786 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:12.784+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:12.865 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:12.863+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:13.765 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:13.763+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:13.882 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:13.879+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:14.814 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:14.812+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:14.843 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:14.841+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:15.781 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:15.778+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:15.867 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:15.865+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:16.794 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:16.792+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:16.913 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:16.911+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:17.746 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:17.744+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:17.931 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:17.929+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:18.796 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:18.794+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:18.909 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:18.907+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:19.828 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:19.826+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:19.883 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:19.881+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:20.820 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:20.817+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:20.865 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:20.862+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:21.832 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:21.830+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:21.847 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:21.845+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:22.832 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:22.830+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:22.874 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:22.872+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:23.826 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:23.823+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:23.860 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:23.858+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:24.841 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:24.797+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:24.892 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:24.890+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:25.840 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:25.838+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:25.903 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:25.900+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:26.856 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:26.854+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:26.861 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:26.859+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:27.824 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:27.821+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:27.907 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:27.904+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:28.818 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:28.816+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:28.878 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:28.876+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:29.811 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:29.809+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:29.869 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:29.866+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:30.830 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:30.827+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:30.855 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:30.853+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:31.848 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:31.846+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:31.861 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:31.859+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:32.810 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:32.808+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:32.835 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:32.833+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:33.801 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:33.799+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:33.802 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:33.800+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:34.752 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:34.750+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:34.821 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:34.818+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:35.755 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:35.752+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:35.846 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:35.843+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:36.760 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:36.757+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:36.852 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:36.850+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:37.762 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:37.760+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:37.856 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:37.853+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:38.770 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:38.766+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:38.831 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:38.829+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:39.807 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:39.804+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:39.825 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:39.823+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:40.800 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:40.797+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:40.863 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:40.861+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:41.794 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:41.792+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:41.890 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:41.888+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:42.792 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:42.790+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:42.911 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:42.909+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:43.801 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:43.799+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:43.873 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:43.871+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:44.777 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:44.775+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:44.828 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:44.826+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:45.821 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:45.819+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:45.860 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:45.859+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:46.807 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:46.806+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:46.878 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:46.877+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:47.849 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:47.848+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:47.892 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:47.891+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:48.805 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:48.804+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:48.868 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:48.866+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:49.783 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:49.782+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:49.905 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:49.904+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:50.746 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:50.745+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:50.962 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:50.941+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:51.747 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:51.746+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:51.976 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:51.975+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:52.752 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:52.751+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:53.014 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:53.013+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:53.725 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:53.724+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:53.968 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:53.967+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:54.731 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:54.730+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:54.972 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:54.970+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:55.708 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:55.707+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:55.983 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:55.982+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:56.695 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:56.694+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:57.010 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:57.008+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:57.716 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:57.715+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:58.000 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:57.999+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:58.742 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:58.741+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:59.044 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:50:59.042+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:50:59.764 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:50:59.763+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:00.082 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:00.081+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:00.813 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:00.811+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:01.038 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:01.035+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:01.822 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:01.819+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:02.020 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:02.017+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:02.808 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:02.805+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:03.019 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:03.016+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:03.857 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:03.854+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:04.038 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:04.036+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:04.810 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:04.808+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:05.074 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:05.072+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:05.851 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:05.848+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:06.114 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:06.111+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:06.844 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:06.842+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:07.148 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:07.146+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:07.822 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:07.820+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:08.139 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:08.137+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:08.818 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:08.816+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:09.182 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:09.180+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:09.833 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:09.830+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:10.228 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:10.226+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:10.858 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:10.855+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:11.270 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:11.268+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:11.819 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:11.816+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:12.270 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:12.268+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:12.814 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:12.812+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:13.269 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:13.267+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:13.848 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:13.845+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:14.277 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:14.275+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:14.869 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:14.866+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:15.260 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:15.258+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:15.891 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:15.889+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:16.288 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:16.286+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:16.863 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:16.861+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:17.256 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:17.254+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:17.904 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:17.902+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:18.301 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:18.299+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:18.914 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:18.912+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:19.334 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:19.332+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:19.927 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:19.925+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:20.358 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:20.356+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:20.918 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:20.916+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:21.378 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:21.376+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:21.947 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:21.945+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:22.368 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:22.365+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:22.954 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:22.952+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:23.350 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:23.348+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:23.936 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:23.933+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:24.339 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:24.337+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:24.953 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:24.950+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:25.329 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:25.327+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:25.984 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:25.981+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:26.290 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:26.288+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:27.000 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:26.998+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:27.283 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:27.281+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:27.961 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:27.959+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:28.318 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:28.316+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:28.966 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:28.964+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:29.280 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:29.277+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:29.935 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:29.933+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:30.302 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:30.300+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:30.966 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:30.964+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:31.298 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:31.296+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:31.930 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:31.928+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:32.256 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:32.254+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:32.884 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:32.882+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:33.274 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:33.273+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:33.914 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:33.913+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:34.272 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:34.270+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:34.893 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:34.892+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:35.282 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:35.281+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:35.908 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:35.907+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:36.291 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:36.290+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:36.879 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:36.878+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:37.265 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:37.264+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:37.874 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:37.873+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:38.256 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:38.254+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:38.828 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:38.826+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:39.292 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:39.291+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:39.786 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:39.785+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:40.278 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:40.277+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:40.767 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:40.766+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:41.257 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:41.256+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:41.730 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:41.729+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:42.274 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:42.273+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:42.746 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:42.744+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:43.288 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:43.287+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:43.774 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:43.772+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:44.309 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:44.307+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:44.750 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:44.749+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:45.349 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:45.347+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:45.719 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:45.718+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:46.304 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:46.303+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:46.673 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:46.670+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:47.260 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:47.258+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:47.720 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:47.719+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:48.225 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:48.224+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:48.715 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:48.713+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:49.232 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:49.230+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:49.698 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:49.697+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:50.190 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:50.189+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:50.725 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:50.723+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:51.215 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:51.213+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:51.745 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:51.743+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:52.168 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:52.165+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:52.762 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:52.760+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:53.178 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:53.176+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:53.797 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:53.795+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:54.144 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:54.142+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:54.768 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:54.766+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:55.116 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:55.113+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:55.787 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:55.784+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:56.086 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:56.084+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:56.749 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:56.746+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:57.064 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:57.061+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:57.761 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:57.758+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:58.031 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:58.028+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:58.742 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:58.740+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:59.000 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:58.998+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:59.706 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:51:59.704+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:51:59.963 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:51:59.960+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:00.734 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:00.731+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:00.928 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:00.926+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:01.771 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:01.769+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:01.906 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:01.904+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:02.781 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:02.778+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:02.938 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:02.936+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:03.732 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:03.729+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:03.960 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:03.958+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:04.711 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:04.709+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:04.964 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:04.962+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:05.706 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:05.704+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:05.980 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:05.978+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:06.711 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:06.709+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:06.944 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:06.942+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:07.730 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:07.728+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:07.906 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:07.904+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:08.752 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:08.750+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:08.924 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:08.922+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:09.723 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:09.721+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:09.974 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:09.971+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:10.770 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:10.768+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:10.980 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:10.978+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:11.736 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:11.733+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:12.005 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:12.003+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:12.715 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:12.713+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:12.963 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:12.961+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:13.672 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:13.670+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:13.941 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:13.938+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:14.682 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:14.680+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:14.896 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:14.893+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:15.689 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:15.686+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:15.942 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:15.939+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:16.692 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:16.690+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:16.980 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:16.977+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:17.712 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:17.710+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:17.981 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:17.979+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:18.727 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:18.724+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:18.942 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:18.940+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:19.712 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:19.710+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:19.904 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:19.902+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:20.708 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:20.705+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:20.911 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:20.909+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:21.686 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:21.683+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:21.868 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:21.866+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:22.703 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:22.701+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:22.865 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:22.864+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:23.664 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:23.663+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:23.830 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:23.828+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:24.691 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:24.690+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:24.793 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:24.791+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:25.704 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:25.702+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:25.770 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:25.768+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:26.668 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:26.666+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:26.808 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:26.807+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:27.634 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:27.633+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:27.789 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:27.788+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:28.633 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:28.631+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:28.822 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:28.821+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:29.633 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:29.632+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:29.778 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:29.777+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:30.585 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:30.584+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:30.782 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:30.780+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:31.606 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:31.604+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:31.735 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:31.734+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:32.597 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:32.595+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:32.720 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:32.719+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:33.558 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:33.557+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:33.702 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:33.701+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:34.556 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:34.554+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:34.680 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:34.679+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:35.578 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:35.577+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:35.642 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:35.640+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:36.626 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:36.624+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:36.676 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:36.675+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:37.671 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:37.670+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:37.700 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:37.698+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:38.706 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:38.702+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:38.725 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:38.724+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:39.743 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:39.741+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:39.762 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:39.761+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:40.724 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:40.723+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:40.793 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:40.790+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:41.681 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:41.678+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:41.776 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:41.774+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:42.650 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:42.648+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:42.812 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:42.810+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:43.629 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:43.626+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:43.840 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:43.838+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:44.675 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:44.672+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:44.799 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:44.796+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:45.713 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:45.710+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:45.781 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:45.778+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:46.748 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:46.745+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:46.784 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:46.782+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:47.782 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:47.779+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:47.810 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:47.808+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:48.768 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:48.765+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:48.834 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:48.832+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:49.803 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:49.801+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:49.812 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:49.810+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:50.834 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:50.832+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:50.836 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:50.834+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:51.859 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:51.857+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:51.882 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:51.879+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:52.836 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:52.833+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:52.851 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:52.848+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:53.802 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:53.799+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:53.865 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:53.863+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:54.778 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:54.776+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:54.913 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:54.910+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:55.825 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:55.823+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:55.925 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:55.923+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:56.791 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:56.788+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:56.886 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:56.883+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:57.750 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:57.748+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:57.898 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:57.896+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:58.730 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:58.727+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:58.916 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:58.913+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:59.770 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:52:59.768+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:52:59.928 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:52:59.926+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:00.791 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:00.789+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:00.903 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:00.900+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:01.754 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:01.752+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:01.870 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:01.868+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:02.763 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:02.760+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:02.832 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:02.830+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:03.730 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:03.728+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:03.862 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:03.859+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:04.716 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:04.714+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:04.888 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:04.886+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:05.701 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:05.698+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:05.888 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:05.885+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:06.675 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:06.672+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:06.906 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:06.904+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:07.718 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:07.716+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:07.947 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:07.944+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:08.766 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:08.764+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:08.930 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:08.928+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:09.718 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:09.716+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:09.975 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:09.972+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:10.743 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:10.740+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:11.003 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:11.000+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:11.743 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:11.741+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:12.050 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:12.048+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:12.790 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:12.787+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:13.015 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:13.013+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:13.826 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:13.825+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:14.000 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:13.999+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:14.862 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:14.860+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:15.005 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:15.004+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:15.889 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:15.888+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:16.029 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:16.027+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:16.923 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:16.922+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:17.010 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:17.008+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:17.898 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:17.897+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:18.059 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:18.057+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:18.892 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:18.890+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:19.070 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:19.069+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:19.884 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:19.882+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:20.055 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:20.053+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:20.887 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:20.886+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:21.035 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:21.033+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:21.849 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:21.848+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:21.993 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:21.992+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:22.833 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:22.832+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:23.010 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:23.009+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:23.810 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:23.808+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:24.032 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:24.031+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:24.841 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:24.838+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:24.997 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:24.996+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:25.854 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:25.852+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:25.987 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:25.986+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:26.899 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:26.898+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:27.011 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:27.009+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:27.922 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:27.920+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:27.992 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:27.990+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:28.912 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:28.910+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:29.024 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:29.023+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:29.958 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:29.957+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:30.002 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:30.001+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:30.939 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:30.937+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:31.027 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:31.024+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:31.890 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:31.888+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:31.984 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:31.982+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:32.927 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:32.925+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:33.000 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:32.998+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:33.956 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:33.953+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:34.023 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:34.021+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:34.987 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:34.985+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:35.069 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:35.066+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:35.993 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:35.991+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:36.033 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:36.031+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:36.960 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:36.958+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:37.069 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:37.067+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:37.961 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:37.959+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:38.055 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:38.052+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:38.968 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:38.965+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:39.047 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:39.044+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:39.991 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:39.988+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:40.002 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:40.000+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:40.967 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:40.965+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:40.994 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:40.991+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:41.967 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:41.965+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:42.030 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:42.027+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:42.929 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:42.927+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:42.998 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:42.996+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:43.942 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:43.940+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:43.965 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:43.963+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:44.941 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:44.938+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:44.957 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:44.955+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:45.920 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:45.918+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:45.964 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:45.962+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:46.875 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:46.873+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:46.947 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:46.944+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:47.877 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:47.875+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:47.931 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:47.929+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:48.870 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:48.867+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:48.914 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:48.911+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:49.862 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:49.860+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:49.932 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:49.929+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:50.836 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:50.833+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:50.930 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:50.927+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:51.883 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:51.881+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:51.899 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:51.896+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:52.866 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:52.864+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:52.886 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:52.884+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:53.848 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:53.845+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:53.923 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:53.921+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:54.883 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:54.880+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:54.888 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:54.885+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:55.841 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:55.839+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:55.933 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:55.931+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:56.842 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:56.839+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:56.898 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:56.896+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:57.870 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:57.866+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:57.875 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:57.872+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:58.851 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:58.848+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:58.903 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:58.901+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:59.826 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:53:59.824+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:53:59.939 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:53:59.937+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:00.875 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:00.872+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:00.955 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:00.952+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:01.871 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:01.869+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:02.004 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:02.001+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:02.873 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:02.870+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:02.989 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:02.984+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:03.868 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:03.867+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:03.987 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:03.985+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:04.851 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:04.849+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:05.026 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:05.008+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:05.876 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:05.875+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:05.970 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:05.969+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:06.874 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:06.873+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:07.012 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:07.010+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:07.830 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:07.829+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:08.026 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:08.025+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:08.831 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:08.829+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:09.064 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:09.063+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:09.862 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:09.861+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:10.074 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:10.072+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:10.887 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:10.886+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:11.081 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:11.079+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:11.914 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:11.913+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:12.098 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:12.097+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:12.956 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:12.955+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:13.136 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:13.135+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:13.956 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:13.955+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:14.179 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:14.178+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:15.002 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:15.001+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:15.162 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:15.160+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:15.991 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:15.989+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:16.127 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:16.126+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:16.982 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:16.981+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:17.083 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:17.081+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:18.003 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:18.001+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:18.034 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:18.032+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:19.019 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:19.017+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:19.034 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:19.033+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:19.994 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:19.992+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:20.008 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:20.006+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:21.004 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:21.003+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:21.050 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:21.049+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:22.004 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:22.002+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:22.078 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:22.075+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:22.976 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:22.973+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:23.034 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:23.032+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:24.017 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:24.015+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:24.070 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:24.067+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:24.991 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:24.989+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:25.089 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:25.086+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:26.036 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:26.034+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:26.102 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:26.100+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:27.015 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:27.013+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:27.058 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:27.055+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:27.992 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:27.989+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:28.042 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:28.040+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:28.985 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:28.982+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:29.004 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:29.001+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:29.947 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:29.944+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:29.968 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:29.966+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:30.946 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:30.943+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:30.991 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:30.988+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:31.919 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:31.916+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:32.015 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:32.013+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:32.887 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:32.884+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:33.010 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:33.007+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:33.838 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:33.836+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:33.963 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:33.960+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:34.854 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:34.852+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:34.951 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:34.949+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:35.819 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:35.816+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:35.944 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:35.942+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:36.864 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:36.861+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:36.952 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:36.950+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:37.847 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:37.845+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:37.955 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:37.952+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:38.802 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:38.799+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:38.948 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:38.946+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:39.832 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:39.830+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:39.933 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:39.930+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:40.789 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:40.787+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:40.890 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:40.888+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:41.751 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:41.749+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:41.887 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:41.885+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:42.755 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:42.752+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:42.846 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:42.843+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:43.762 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:43.760+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:43.850 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:43.847+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:44.784 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:44.782+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:44.851 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:44.848+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:45.819 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:45.816+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:45.899 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:45.897+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:46.822 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:46.820+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:46.915 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:46.913+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:47.783 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:47.780+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:47.952 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:47.949+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:48.826 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:48.823+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:48.992 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:48.989+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:49.790 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:49.787+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:49.993 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:49.990+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:50.777 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:50.775+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:50.978 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:50.976+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:51.737 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:51.735+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:51.951 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:51.948+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:52.779 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:52.777+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:52.981 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:52.979+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:53.773 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:53.771+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:54.010 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:54.008+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:54.759 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:54.757+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:55.010 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:55.009+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:55.775 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:55.774+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:56.049 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:56.047+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:56.810 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:56.808+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:57.094 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:57.092+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:57.830 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:57.829+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:58.123 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:58.121+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:58.843 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:58.841+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:59.130 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:54:59.128+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:54:59.888 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:54:59.886+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:00.120 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:00.119+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:00.936 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:00.934+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:01.090 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:01.088+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:01.897 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:01.896+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:02.077 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:02.076+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:02.879 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:02.878+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:03.065 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:03.064+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:03.856 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:03.854+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:04.086 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:04.084+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:04.816 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:04.814+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:05.059 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:05.057+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:05.832 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:05.830+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:06.067 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:06.065+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:06.815 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:06.814+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:07.087 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:07.085+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:07.797 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:07.795+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:08.119 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:08.118+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:08.842 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:08.841+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:09.096 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:09.094+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:09.810 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:09.808+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:10.052 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:10.051+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:10.831 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:10.829+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:11.016 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:11.015+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:11.828 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:11.825+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:12.057 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:12.055+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:12.833 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:12.830+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:13.079 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:13.076+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:13.879 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:13.876+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:14.113 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:14.110+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:14.855 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:14.853+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:15.131 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:15.128+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:15.862 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:15.859+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:16.181 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:16.179+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:16.826 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:16.824+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:17.141 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:17.138+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:17.823 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:17.820+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:18.177 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:18.175+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:18.777 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:18.774+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:19.174 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:19.171+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:19.730 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:19.727+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:20.151 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:20.149+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:20.681 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:20.679+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:21.166 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:21.162+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:21.671 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:21.669+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:22.123 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:22.120+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:22.700 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:22.698+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:23.142 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:23.139+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:23.654 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:23.651+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:24.134 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:24.132+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:24.662 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:24.659+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:25.103 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:25.100+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:25.710 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:25.708+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:26.143 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:26.140+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:26.747 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:26.744+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:27.101 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:27.097+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:27.710 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:27.707+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:28.094 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:28.092+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:28.748 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:28.745+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:29.085 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:29.083+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:29.733 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:29.731+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:30.102 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:30.099+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:30.776 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:30.773+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:31.105 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:31.102+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:31.788 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:31.786+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:32.072 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:32.069+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:32.794 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:32.792+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:33.051 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:33.048+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:33.782 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:33.779+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:34.090 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:34.087+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:34.747 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:34.745+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:35.063 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:35.060+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:35.715 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:35.713+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:36.090 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:36.088+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:36.702 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:36.699+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:37.063 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:37.060+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:37.660 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:37.657+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:38.020 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:38.017+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:38.698 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:38.695+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:39.020 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:39.018+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:39.653 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:39.651+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:40.018 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:40.015+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:40.612 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:40.609+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:40.970 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:40.968+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:41.613 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:41.610+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:41.957 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:41.954+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:42.608 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:42.605+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:42.927 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:42.925+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:43.645 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:43.643+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:43.903 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:43.901+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:44.598 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:44.597+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:44.940 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:44.939+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:45.549 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:45.548+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:45.942 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:45.941+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:46.503 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:46.501+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:46.895 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:46.894+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:47.486 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:47.484+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:47.871 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:47.869+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:48.492 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:48.491+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:48.910 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:48.909+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:49.537 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:49.535+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:49.928 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:49.926+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:50.546 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:50.545+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:50.888 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:50.887+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:51.568 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:51.566+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:51.889 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:51.886+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:52.525 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:52.523+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:52.859 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:52.858+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:53.523 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:53.522+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:53.839 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:53.838+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:54.558 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:54.556+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:54.790 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:54.789+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:55.515 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:55.514+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:55.749 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:55.747+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:56.470 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:56.468+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:56.705 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:56.703+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:57.433 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:57.431+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:57.664 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:57.662+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:58.416 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:58.415+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:58.666 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:58.664+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:59.445 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:55:59.444+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:55:59.634 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:55:59.633+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:00.490 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:00.489+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:00.608 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:00.606+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:01.476 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:01.475+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:01.627 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:01.624+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:02.517 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:02.514+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:02.626 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:02.623+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:03.558 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:03.555+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:03.662 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:03.660+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:04.513 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:04.511+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:04.667 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:04.664+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:05.523 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:05.521+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:05.657 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:05.655+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:06.533 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:06.530+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:06.671 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:06.668+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:07.530 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:07.527+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:07.629 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:07.626+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:08.548 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:08.546+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:08.626 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:08.623+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:09.538 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:09.535+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:09.602 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:09.600+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:10.551 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:10.548+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:10.601 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:10.598+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:11.511 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:11.509+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:11.649 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:11.646+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:12.554 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:12.552+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:12.650 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:12.648+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:13.572 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:13.569+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:13.665 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:13.662+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:14.622 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:14.620+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:14.633 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:14.631+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:15.595 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:15.592+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:15.655 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:15.653+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:16.638 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:16.636+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:16.703 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:16.701+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:17.634 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:17.632+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:17.703 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:17.700+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:18.639 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:18.636+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:18.735 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:18.733+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:19.597 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:19.592+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:19.725 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:19.723+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:20.602 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:20.600+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:20.698 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:20.695+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:21.632 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:21.630+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:21.704 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:21.702+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:22.588 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:22.585+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:22.669 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:22.667+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:23.610 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:23.608+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:23.649 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:23.646+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:24.631 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:24.628+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:24.680 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:24.677+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:25.620 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:25.618+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:25.686 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:25.683+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:26.640 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:26.638+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:26.731 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:26.728+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:27.593 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:27.590+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:27.693 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:27.691+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:28.576 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:28.574+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:28.699 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:28.697+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:29.623 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:29.621+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:29.729 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:29.727+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:30.635 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:30.633+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:30.773 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:30.771+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:31.671 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:31.669+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:31.792 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:31.789+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:32.674 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:32.671+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:32.772 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:32.769+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:33.657 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:33.655+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:33.823 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:33.820+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:34.684 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:34.682+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:34.865 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:34.864+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:35.654 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:35.652+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:35.887 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:35.886+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:36.639 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:36.638+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:36.884 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:36.883+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:37.676 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:37.674+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:37.897 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:37.896+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:38.680 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:38.678+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:38.935 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:38.933+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:39.700 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:39.699+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:39.952 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:39.951+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:40.658 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:40.657+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:40.969 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:40.967+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:41.654 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:41.652+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:41.991 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:41.989+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:42.691 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:42.689+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:43.038 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:43.037+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:43.700 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:43.699+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:44.057 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:44.055+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:44.713 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:44.712+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:45.072 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:45.070+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:45.666 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:45.664+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:46.073 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:46.072+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:46.641 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:46.640+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:47.119 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:47.117+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:47.671 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:47.670+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:48.116 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:48.115+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:48.680 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:48.678+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:49.102 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:49.100+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:49.716 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:49.714+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:50.143 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:50.142+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:50.740 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:50.739+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:51.157 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:51.155+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:51.721 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:51.718+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:52.175 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:52.172+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:52.727 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:52.725+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:53.140 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:53.137+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:53.740 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:53.737+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:54.154 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:54.151+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:54.730 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:54.728+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:55.204 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:55.202+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:55.768 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:55.765+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:56.214 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:56.212+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:56.750 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:56.748+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:57.185 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:57.182+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:57.760 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:57.757+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:58.149 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:58.145+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:58.765 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:58.760+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:59.136 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:56:59.134+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:56:59.731 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:56:59.728+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:00.138 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:00.136+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:00.716 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:00.713+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:01.154 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:01.152+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:01.687 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:01.684+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:02.184 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:02.181+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:02.720 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:02.718+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:03.205 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:03.203+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:03.756 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:03.753+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:04.196 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:04.193+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:04.797 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:04.794+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:05.220 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:05.218+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:05.826 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:05.824+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:06.235 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:06.233+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:06.852 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:06.849+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:07.241 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:07.238+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:07.844 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:07.842+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:08.209 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:08.205+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:08.802 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:08.799+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:09.230 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:09.228+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:09.797 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:09.795+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:10.205 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:10.202+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:10.804 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:10.801+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:11.197 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:11.194+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:11.770 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:11.768+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:12.165 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:12.162+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:12.779 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:12.776+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:13.153 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:13.151+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:13.784 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:13.781+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:14.164 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:14.161+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:14.752 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:14.749+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:15.189 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:15.186+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:15.742 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:15.740+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:16.211 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:16.208+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:16.754 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:16.752+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:17.241 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:17.239+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:17.804 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:17.802+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:18.201 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:18.198+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:18.850 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:18.847+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:19.245 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:19.243+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:19.863 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:19.860+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:20.282 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:20.280+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:20.895 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:20.892+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:21.271 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:21.269+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:21.908 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:21.905+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:22.269 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:22.267+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:22.921 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:22.918+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:23.250 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:23.247+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:23.895 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:23.892+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:24.262 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:24.259+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:24.846 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:24.844+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:25.275 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:25.274+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:25.806 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:25.805+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:26.263 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:26.261+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:26.788 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:26.786+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:27.252 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:27.251+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:27.774 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:27.772+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:28.293 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:28.292+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:28.786 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:28.785+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:29.311 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:29.309+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:29.761 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:29.759+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:30.280 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:30.278+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:30.758 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:30.756+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:31.264 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:31.263+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:31.757 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:31.755+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:32.219 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:32.217+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:32.795 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:32.793+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:33.192 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:33.190+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:33.837 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:33.835+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:34.177 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:34.175+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:34.858 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:34.856+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:35.171 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:35.170+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:35.863 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:35.861+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:36.161 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:36.160+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:36.858 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:36.857+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:37.172 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:37.170+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:37.825 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:37.823+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:38.205 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:38.203+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:38.873 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:38.871+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:39.181 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:39.180+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:39.918 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:39.916+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:40.187 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:40.185+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:40.909 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:40.907+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:41.158 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:41.156+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:41.905 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:41.902+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:42.202 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:42.199+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:42.870 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:42.867+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:43.235 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:43.232+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:43.840 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:43.837+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:44.191 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:44.189+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:44.829 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:44.826+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:45.221 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:45.218+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:45.878 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:45.875+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:46.180 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:46.177+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:46.836 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:46.833+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:47.162 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:47.159+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:47.882 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:47.879+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:48.150 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:48.147+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:48.915 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:48.912+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:49.127 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:49.124+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:49.945 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:49.943+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:50.147 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:50.144+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:50.950 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:50.947+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:51.153 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:51.150+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:51.976 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:51.974+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:52.105 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:52.103+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:52.963 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:52.960+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:53.112 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:53.109+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:53.989 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:53.986+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:54.090 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:54.087+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:55.030 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:55.027+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:55.047 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:55.044+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:56.022 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:56.019+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:56.027 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:56.025+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:57.019 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:57.013+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:57.067 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:57.065+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:58.013 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:58.011+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:58.075 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:58.072+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:59.019 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:57:59.017+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:57:59.090 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:57:59.087+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:00.010 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:00.008+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:00.111 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:00.108+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:01.018 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:01.015+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:01.087 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:01.085+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:02.047 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:02.044+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:02.085 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:02.083+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:03.034 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:03.031+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:03.050 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:03.047+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:04.040 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:04.038+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:04.083 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:04.080+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:05.016 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:05.013+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:05.053 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:05.051+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:06.015 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:06.012+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:06.096 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:06.093+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:06.979 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:06.976+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:07.137 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:07.134+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:07.965 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:07.962+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:08.148 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:08.146+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:08.968 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:08.965+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:09.144 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:09.142+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:09.956 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:09.953+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:10.116 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:10.113+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:10.991 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:10.988+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:11.151 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:11.148+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:12.015 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:12.012+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:12.128 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:12.126+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:13.059 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:13.056+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:13.171 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:13.169+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:14.080 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:14.079+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:14.157 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:14.155+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:15.050 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:15.048+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:15.195 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:15.194+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:16.045 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:16.043+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:16.200 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:16.198+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:17.085 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:17.083+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:17.203 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:17.202+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:18.134 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:18.132+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:18.250 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:18.249+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:19.182 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:19.180+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:19.244 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:19.242+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:20.167 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:20.165+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:20.218 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:20.216+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:21.144 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:21.142+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:21.178 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:21.176+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:22.161 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:22.159+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:22.164 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:22.162+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:23.117 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:23.115+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:23.163 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:23.161+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:24.114 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:24.112+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:24.156 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:24.154+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:25.072 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:25.070+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:25.125 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:25.123+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:26.024 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:26.023+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:26.095 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:26.093+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:27.005 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:27.004+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:27.082 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:27.081+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:28.047 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:28.045+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:28.061 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:28.059+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:29.038 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:29.035+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:29.062 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:29.061+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:30.009 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:30.007+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:30.112 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:30.110+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:30.964 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:30.962+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:31.092 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:31.090+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:31.962 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:31.961+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:32.122 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:32.119+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:32.959 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:32.955+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:33.092 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:33.089+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:33.938 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:33.935+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:34.104 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:34.102+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:34.916 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:34.914+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:35.094 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:35.091+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:35.881 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:35.878+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:36.067 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:36.064+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:36.883 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:36.881+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:37.114 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:37.111+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:37.890 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:37.888+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:38.080 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:38.077+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:38.842 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:38.839+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:39.061 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:39.059+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:39.837 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:39.834+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:40.082 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:40.080+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:40.835 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:40.833+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:41.069 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:41.066+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:41.794 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:41.790+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:42.082 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:42.079+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:42.816 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:42.813+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:43.060 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:43.058+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:43.862 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:43.860+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:44.110 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:44.108+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:44.864 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:44.861+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:45.103 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:45.101+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:45.846 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:45.843+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:46.103 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:46.101+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:46.830 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:46.828+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:47.151 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:47.149+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:47.794 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:47.791+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:48.145 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:48.143+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:48.824 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:48.821+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:49.127 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:49.124+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:49.862 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:49.859+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:50.163 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:50.160+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:50.832 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:50.830+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:51.158 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:51.155+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:51.829 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:51.826+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:52.193 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:52.190+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:52.875 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:52.873+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:53.156 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:53.154+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:53.911 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:53.908+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:54.199 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:54.196+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:54.955 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:54.952+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:55.154 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:55.152+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:55.923 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:55.920+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:56.197 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:56.194+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:56.909 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:56.907+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:57.184 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:57.182+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:57.905 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:57.902+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:58.161 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:58.159+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:58.861 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:58.858+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:59.125 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:58:59.122+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:58:59.907 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:58:59.905+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:00.136 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:00.133+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:00.910 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:00.907+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:01.164 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:01.162+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:01.947 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:01.945+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:02.182 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:02.179+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:02.951 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:02.948+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:03.138 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:03.136+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:03.910 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:03.907+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:04.112 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:04.110+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:04.901 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:04.899+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:05.156 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:05.155+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:05.931 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:05.929+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:06.151 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:06.150+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:06.958 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:06.956+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:07.158 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:07.156+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:07.973 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:07.971+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:08.190 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:08.188+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:08.955 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:08.954+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:09.220 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:09.218+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:09.929 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:09.927+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:10.246 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:10.244+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:10.970 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:10.968+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:11.221 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:11.219+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:11.962 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:11.960+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:12.258 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:12.256+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:12.959 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:12.957+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:13.301 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:13.300+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:13.965 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:13.964+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:14.314 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:14.312+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:14.982 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:14.980+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:15.353 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:15.352+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:15.941 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:15.939+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:16.327 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:16.326+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:16.899 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:16.897+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:17.369 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:17.368+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:17.860 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:17.858+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:18.370 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:18.369+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:18.828 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:18.827+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:19.421 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:19.419+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:19.863 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:19.861+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:20.410 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:20.409+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:20.879 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:20.877+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:21.439 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:21.438+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:21.884 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:21.882+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:22.470 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:22.468+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:22.928 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:22.926+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:23.448 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:23.445+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:23.930 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:23.927+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:24.473 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:24.470+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:24.914 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:24.911+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:25.502 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:25.499+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:25.947 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:25.944+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:26.525 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:26.522+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:26.976 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:26.973+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:27.513 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:27.510+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:27.979 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:27.977+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:28.465 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:28.462+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:29.024 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:29.021+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:29.512 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:29.509+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:30.028 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:30.025+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:30.553 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:30.550+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:31.003 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:31.000+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:31.555 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:31.552+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:32.011 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:32.009+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:32.554 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:32.551+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:33.019 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:33.016+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:33.553 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:33.550+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:34.032 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:34.029+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:34.532 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:34.529+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:35.075 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:35.072+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:35.527 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:35.525+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:36.076 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:36.073+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:36.575 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:36.573+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:37.030 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:37.027+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:37.569 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:37.567+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:38.058 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:38.055+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:38.568 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:38.566+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:39.051 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:39.049+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:39.525 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:39.522+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:40.077 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:40.075+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:40.545 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:40.543+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:41.029 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:41.026+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:41.524 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:41.521+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:42.075 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:42.072+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:42.497 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:42.495+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:43.090 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:43.087+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:43.521 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:43.519+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:44.106 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:44.104+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:44.517 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:44.515+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:45.116 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:45.113+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:45.507 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:45.504+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:46.093 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:46.091+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:46.520 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:46.517+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:47.052 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:47.049+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:47.507 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:47.504+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:48.014 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:48.012+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:48.534 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:48.531+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:49.000 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:48.997+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:49.528 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:49.526+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:49.977 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:49.974+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:50.567 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:50.564+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:50.979 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:50.976+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:51.536 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:51.534+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:51.958 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:51.955+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:52.533 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:52.530+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:52.953 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:52.951+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:53.540 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:53.537+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:53.977 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:53.975+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:54.494 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:54.492+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:54.939 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:54.938+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:55.472 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:55.469+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:55.940 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:55.938+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:56.464 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:56.462+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:56.916 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:56.915+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:57.475 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:57.473+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:57.869 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:57.867+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:58.507 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:58.505+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:58.824 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:58.823+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:59.534 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T18:59:59.532+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T18:59:59.869 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T18:59:59.868+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:00.525 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:00.522+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:00.882 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:00.880+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:01.527 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:01.526+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:01.900 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:01.898+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:02.525 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:02.524+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:02.870 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:02.868+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:03.505 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:03.503+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:03.884 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:03.882+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:04.547 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:04.545+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:04.846 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:04.844+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:05.506 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:05.504+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:05.883 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:05.881+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:06.457 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:06.456+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:06.852 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:06.851+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:07.468 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:07.466+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:07.866 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:07.864+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:08.461 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:08.460+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:08.829 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:08.827+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:09.421 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:09.419+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:09.865 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:09.864+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:10.406 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:10.404+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:10.825 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:10.824+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:11.436 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:11.435+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:11.785 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:11.781+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:12.429 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:12.426+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:12.809 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:12.807+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:13.388 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:13.385+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:13.788 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:13.786+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:14.418 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:14.415+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:14.818 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:14.815+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:15.444 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:15.441+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:15.812 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:15.809+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:16.418 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:16.415+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:16.832 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:16.830+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:17.400 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:17.397+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:17.842 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:17.840+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:18.375 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:18.372+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:18.882 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:18.879+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:19.368 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:19.365+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:19.847 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-10-18T19:00:19.848 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T19:00:19.845+0000 7fd626ffd700 -1 Fail to open '/proc/91154/cmdline' error = (2) No such file or directory 2023-10-18T19:00:19.848 INFO:tasks.workunit.client.0.smithi082.stderr:2023-10-18T19:00:19.845+0000 7fd626ffd700 -1 received signal: Terminated from (PID: 91154) UID: 1000 2023-10-18T19:00:19.881 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:19.878+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:20.336 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:20.333+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:20.884 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:20.881+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:21.290 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:21.287+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:21.861 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:21.858+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:22.316 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:22.313+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:22.816 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:22.814+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:23.299 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:23.296+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:23.803 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:23.800+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:24.335 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:24.333+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:24.795 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:24.792+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:25.306 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:25.303+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:25.788 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:25.786+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:26.331 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:26.329+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:26.771 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:26.768+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:27.372 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:27.369+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:27.791 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:27.788+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:28.389 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:28.387+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:28.789 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:28.786+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:29.387 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:29.384+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:29.776 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:29.774+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:30.379 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:30.376+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:30.802 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:30.800+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:31.332 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:31.329+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:31.828 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:31.825+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:32.307 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:32.305+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:32.876 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:32.873+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:33.345 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:33.343+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:33.900 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:33.897+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:34.325 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:34.322+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:34.950 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:34.947+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:35.277 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:35.274+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:35.978 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:35.975+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:36.257 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:36.254+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:36.987 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:36.984+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:37.262 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:37.259+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:38.026 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:38.023+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:38.214 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:38.212+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:39.008 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:39.005+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:39.194 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:39.191+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:40.026 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:40.024+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:40.194 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:40.191+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:41.077 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:41.074+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:41.203 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:41.201+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:42.098 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:42.095+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:42.240 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:42.237+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:43.108 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:43.105+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:43.263 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:43.260+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:44.087 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:44.084+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:44.272 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:44.269+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:45.125 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:45.124+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:45.253 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:45.252+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:46.125 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:46.124+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:46.221 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:46.219+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:47.109 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:47.108+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:47.255 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:47.254+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:48.082 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:48.080+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:48.241 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:48.239+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:49.040 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:49.038+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:49.221 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:49.219+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:49.996 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:49.994+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:50.247 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:50.245+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:51.045 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:51.043+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:51.252 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:51.250+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:52.010 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:52.008+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:52.217 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:52.215+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:53.025 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:53.023+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:53.169 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:53.168+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:53.986 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:53.984+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:54.132 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:54.130+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:54.992 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:54.990+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:55.087 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:55.086+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:55.949 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:55.947+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:56.065 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:56.064+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:56.913 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:56.911+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:57.042 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:57.040+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:57.898 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:57.896+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:58.088 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:58.086+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:58.899 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:58.897+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:59.040 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:00:59.039+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:00:59.903 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:00:59.901+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:00.043 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:00.041+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:00.907 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:00.906+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:01.034 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:01.032+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:01.940 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:01.938+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:02.057 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:02.055+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:02.941 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:02.938+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:03.039 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:03.037+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:03.962 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:03.959+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:04.058 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:04.055+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:04.976 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:04.973+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:05.089 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:05.087+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:05.943 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:05.941+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:06.109 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:06.106+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:06.894 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:06.891+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:07.110 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:07.107+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:07.920 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:07.917+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:08.142 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:08.139+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:08.906 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:08.903+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:09.175 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:09.173+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:09.956 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:09.954+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:10.177 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:10.174+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:10.909 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:10.906+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:11.212 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:11.209+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:11.956 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:11.953+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:12.224 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:12.221+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:12.970 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:12.967+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:13.215 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:13.213+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:13.949 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:13.946+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:14.198 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:14.193+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:14.945 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:14.942+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:15.169 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:15.167+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:15.976 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:15.973+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:16.217 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:16.214+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:16.983 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:16.981+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:17.236 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:17.233+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:18.032 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:18.029+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:18.284 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:18.281+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:19.055 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:19.052+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:19.247 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:19.244+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:19.849 DEBUG:teuthology.orchestra.run:timed out waiting; will kill: 2023-10-18T19:01:20.021 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:20.018+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:20.245 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:20.243+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:21.057 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:21.054+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:21.258 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:21.255+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:22.069 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:22.067+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:22.255 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:22.252+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:23.061 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:23.058+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:23.209 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:23.207+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:24.014 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:24.011+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:24.249 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:24.246+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:25.008 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:25.005+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:25.294 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:25.292+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:26.048 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:26.045+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:26.246 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:26.243+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:27.037 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:27.034+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:27.230 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:27.227+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:28.035 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:28.033+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:28.217 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:28.214+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:29.020 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:29.018+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:29.177 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:29.175+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:30.024 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:30.021+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:30.167 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:30.164+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:31.025 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:31.023+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:31.119 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:31.116+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:32.066 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:32.063+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:32.136 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:32.133+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:33.033 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:33.030+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:33.179 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:33.177+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:34.018 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:34.015+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:34.211 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:34.208+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:35.030 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:35.028+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:35.237 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:35.236+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:36.070 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:36.068+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:36.220 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:36.218+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:37.075 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:37.073+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:37.202 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:37.200+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:38.069 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:38.067+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:38.199 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:38.198+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:39.079 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:39.077+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:39.184 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:39.182+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:40.073 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:40.071+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:40.163 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:40.162+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:41.110 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:41.108+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:41.214 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:41.212+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:42.078 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:42.076+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:42.263 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:42.262+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:43.050 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:43.049+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:43.266 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:43.264+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:44.021 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:44.018+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:44.273 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:44.271+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:45.044 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:45.042+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:45.270 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:45.268+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:46.077 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:46.075+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:46.299 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:46.297+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:47.056 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:47.054+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:47.255 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:47.253+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:48.046 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:48.044+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:48.298 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:48.296+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:49.077 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:49.075+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:49.322 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:49.321+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:50.031 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:50.030+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:50.298 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:50.297+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:51.076 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:51.074+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:51.298 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:51.296+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:52.038 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:52.036+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:52.283 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:52.280+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:53.083 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:53.080+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:53.263 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:53.260+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:54.092 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:54.089+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:54.244 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:54.241+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:55.140 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:55.138+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:55.291 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:55.288+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:56.188 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:56.185+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:56.334 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:56.332+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:57.138 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:57.136+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:57.320 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:57.317+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:58.152 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:58.149+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:58.358 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:58.355+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:59.182 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:01:59.179+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:01:59.406 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:01:59.403+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:00.138 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:00.132+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:00.403 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:00.400+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:01.124 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:01.122+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:01.379 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:01.377+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:02.157 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:02.154+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:02.363 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:02.360+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:03.178 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:03.175+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:03.407 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:03.404+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:04.173 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:04.170+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:04.363 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:04.361+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:05.123 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:05.121+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:05.349 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:05.346+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:06.098 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:06.095+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:06.360 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:06.358+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:07.115 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:07.112+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:07.388 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:07.386+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:08.099 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:08.096+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:08.423 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:08.421+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:09.116 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:09.113+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:09.406 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:09.403+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:10.099 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:10.096+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:10.438 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:10.435+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:11.136 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:11.133+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:11.487 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:11.484+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:12.113 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:12.110+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:12.534 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:12.531+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:13.116 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:13.113+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:13.549 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:13.546+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:14.151 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:14.148+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:14.509 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:14.506+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:15.102 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:15.099+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:15.549 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:15.546+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:16.153 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:16.150+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:16.528 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:16.526+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:17.174 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:17.171+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:17.481 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:17.479+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:18.166 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:18.163+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:18.464 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:18.461+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:19.181 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:19.178+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:19.447 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:19.444+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:19.851 DEBUG:teuthology.orchestra.run:timed out waiting; will kill: 2023-10-18T19:02:19.852 INFO:tasks.workunit:Stopping ['rbd/cli_generic.sh'] on client.0... 2023-10-18T19:02:19.853 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-10-18T19:02:20.138 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_84698e743199450ee5f6969d22ebe6bed3a68e4a/qa/tasks/workunit.py", line 129, in task p.spawn(_run_tests, ctx, refspec, role, tests, File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_84698e743199450ee5f6969d22ebe6bed3a68e4a/qa/tasks/workunit.py", line 423, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test rbd/cli_generic.sh) on smithi082 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=84698e743199450ee5f6969d22ebe6bed3a68e4a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/cli_generic.sh' 2023-10-18T19:02:21.526 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=a93b79dd5eb549ab8f42c4f9f54257fd Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_84698e743199450ee5f6969d22ebe6bed3a68e4a/qa/tasks/workunit.py", line 129, in task p.spawn(_run_tests, ctx, refspec, role, tests, File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_84698e743199450ee5f6969d22ebe6bed3a68e4a/qa/tasks/workunit.py", line 423, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test rbd/cli_generic.sh) on smithi082 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=84698e743199450ee5f6969d22ebe6bed3a68e4a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/cli_generic.sh' 2023-10-18T19:02:21.539 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-18T19:02:21.552 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-18T19:02:21.553 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T19:02:21.558 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:20.205+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:21.559 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:20.481+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:21.560 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:21.221+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:21.560 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:21.456+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:21.863 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:02:21.864 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":154,"fsid":"cb63dacd-a4a9-467a-b30e-f9493a2182c3","created":"2023-10-18T15:59:07.909789+0000","modified":"2023-10-18T17:02:33.449830+0000","last_up_change":"2023-10-18T15:59:15.884655+0000","last_in_change":"2023-10-18T15:59:08.812513+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-10-18T15:59:15.923203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T15:59:16.676775+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"15","last_force_op_resend_prenautilus":"15","last_force_op_resend_preluminous":"15","auid":0,"snap_mode":"selfmanaged","snap_seq":91,"snap_epoch":150,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cache","create_time":"2023-10-18T15:59:24.096546+0000","flags":8201,"flags_names":"hashpspool,incomplete_clones,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"15","last_force_op_resend_prenautilus":"15","last_force_op_resend_preluminous":"15","auid":0,"snap_mode":"selfmanaged","snap_seq":91,"snap_epoch":150,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":2,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"f646c787-fe92-4e2d-abce-6b81d0e6e284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":87291},{"type":"v1","addr":"172.21.15.82:6817","nonce":87291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":87291},{"type":"v1","addr":"172.21.15.82:6819","nonce":87291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":87291},{"type":"v1","addr":"172.21.15.82:6823","nonce":87291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":87291},{"type":"v1","addr":"172.21.15.82:6821","nonce":87291}]},"public_addr":"172.21.15.82:6817/87291","cluster_addr":"172.21.15.82:6819/87291","heartbeat_back_addr":"172.21.15.82:6823/87291","heartbeat_front_addr":"172.21.15.82:6821/87291","state":["exists","up"]},{"osd":1,"uuid":"46810abb-ff37-4aeb-b272-7610274b1957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":87261},{"type":"v1","addr":"172.21.15.82:6809","nonce":87261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":87261},{"type":"v1","addr":"172.21.15.82:6811","nonce":87261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":87261},{"type":"v1","addr":"172.21.15.82:6815","nonce":87261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":87261},{"type":"v1","addr":"172.21.15.82:6813","nonce":87261}]},"public_addr":"172.21.15.82:6809/87261","cluster_addr":"172.21.15.82:6811/87261","heartbeat_back_addr":"172.21.15.82:6815/87261","heartbeat_front_addr":"172.21.15.82:6813/87261","state":["exists","up"]},{"osd":2,"uuid":"a5410593-60ce-4cb3-901e-e96645dc2c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":87262},{"type":"v1","addr":"172.21.15.82:6801","nonce":87262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":87262},{"type":"v1","addr":"172.21.15.82:6803","nonce":87262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":87262},{"type":"v1","addr":"172.21.15.82:6807","nonce":87262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":87262},{"type":"v1","addr":"172.21.15.82:6805","nonce":87262}]},"public_addr":"172.21.15.82:6801/87262","cluster_addr":"172.21.15.82:6803/87262","heartbeat_back_addr":"172.21.15.82:6807/87262","heartbeat_front_addr":"172.21.15.82:6805/87262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T15:59:14.031532+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T15:59:13.937892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T15:59:13.949251+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":4,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]},{"pool":5,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]},{"pool":7,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T19:02:21.877 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-18T19:02:21.877 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:02:22.168 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:02:22.170 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:02:22.181 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5644,"stamp":"2023-10-18T19:02:20.491194+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25628,"ondisk_log_size":25628,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001593"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"151'1359","reported_seq":2175,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:29.515853+0000","last_change":"2023-10-18T16:03:52.115363+0000","last_active":"2023-10-18T17:02:29.515853+0000","last_peered":"2023-10-18T17:02:29.515853+0000","last_clean":"2023-10-18T17:02:29.515853+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T17:02:29.515853+0000","last_undegraded":"2023-10-18T17:02:29.515853+0000","last_fullsized":"2023-10-18T17:02:29.515853+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1359,"ondisk_log_size":1359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":151,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451303+0000","last_change":"2023-10-18T15:59:17.900205+0000","last_active":"2023-10-18T17:02:28.451303+0000","last_peered":"2023-10-18T17:02:28.451303+0000","last_clean":"2023-10-18T17:02:28.451303+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T17:02:28.451303+0000","last_undegraded":"2023-10-18T17:02:28.451303+0000","last_fullsized":"2023-10-18T17:02:28.451303+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"151'983","reported_seq":1818,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:29.515855+0000","last_change":"2023-10-18T16:03:52.115383+0000","last_active":"2023-10-18T17:02:29.515855+0000","last_peered":"2023-10-18T17:02:29.515855+0000","last_clean":"2023-10-18T17:02:29.515855+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T17:02:29.515855+0000","last_undegraded":"2023-10-18T17:02:29.515855+0000","last_fullsized":"2023-10-18T17:02:29.515855+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":983,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1993,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452064+0000","last_change":"2023-10-18T16:03:52.125315+0000","last_active":"2023-10-18T17:02:28.452064+0000","last_peered":"2023-10-18T17:02:28.452064+0000","last_clean":"2023-10-18T17:02:28.452064+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T17:02:28.452064+0000","last_undegraded":"2023-10-18T17:02:28.452064+0000","last_fullsized":"2023-10-18T17:02:28.452064+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8455,"ondisk_log_size":8455,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740569,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.224,"5min":0.23400000000000001,"15min":0.22900000000000001},"min":{"1min":0.14299999999999999,"5min":0.14299999999999999,"15min":0.14299999999999999},"max":{"1min":0.41199999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.26600000000000001,"15min":0.26200000000000001},"min":{"1min":0.21299999999999999,"5min":0.16,"15min":0.16},"max":{"1min":0.44700000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.28599999999999998}]},{"osd":1,"last update":"Wed Oct 18 19:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.23799999999999999,"15min":0.23699999999999999},"min":{"1min":0.182,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.38200000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.245,"5min":0.25,"15min":0.245},"min":{"1min":0.19400000000000001,"5min":0.16900000000000001,"15min":0.16400000000000001},"max":{"1min":0.42099999999999999,"5min":0.48199999999999998,"15min":0.48599999999999999},"last":0.25700000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740570,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.20599999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"min":{"1min":0.16,"5min":0.153,"15min":0.14099999999999999},"max":{"1min":0.34799999999999998,"5min":0.42499999999999999,"15min":0.436},"last":0.246},{"interface":"front","average":{"1min":0.224,"5min":0.22600000000000001,"15min":0.22800000000000001},"min":{"1min":0.17599999999999999,"5min":0.17299999999999999,"15min":0.16800000000000001},"max":{"1min":0.28599999999999998,"5min":0.433,"15min":0.433},"last":0.27600000000000002}]},{"osd":2,"last update":"Wed Oct 18 19:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.217,"5min":0.22500000000000001,"15min":0.22600000000000001},"min":{"1min":0.184,"5min":0.16700000000000001,"15min":0.154},"max":{"1min":0.29799999999999999,"5min":0.44700000000000001,"15min":0.47599999999999998},"last":0.224},{"interface":"front","average":{"1min":0.246,"5min":0.245,"15min":0.245},"min":{"1min":0.215,"5min":0.17599999999999999,"15min":0.17299999999999999},"max":{"1min":0.311,"5min":0.45800000000000002,"15min":0.45800000000000002},"last":0.28699999999999998}]}]},{"osd":1,"up_from":8,"seq":34359740569,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.224,"5min":0.21099999999999999,"15min":0.20999999999999999},"min":{"1min":0.152,"5min":0.13800000000000001,"15min":0.13800000000000001},"max":{"1min":0.41099999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"last":0.17199999999999999},{"interface":"front","average":{"1min":0.25,"5min":0.24099999999999999,"15min":0.23999999999999999},"min":{"1min":0.18099999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.44,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.21099999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.25900000000000001,"5min":0.23599999999999999,"15min":0.23100000000000001},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.16300000000000001},"max":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"last":0.20300000000000001},{"interface":"front","average":{"1min":0.27100000000000002,"5min":0.26100000000000001,"15min":0.255},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.17000000000000001},"max":{"1min":0.46300000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"last":0.245}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:02:22.183 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:02:22.196 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:22.193+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:22.474 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:02:22.476 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:02:22.487 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5644,"stamp":"2023-10-18T19:02:20.491194+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25628,"ondisk_log_size":25628,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001593"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"151'1359","reported_seq":2175,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:29.515853+0000","last_change":"2023-10-18T16:03:52.115363+0000","last_active":"2023-10-18T17:02:29.515853+0000","last_peered":"2023-10-18T17:02:29.515853+0000","last_clean":"2023-10-18T17:02:29.515853+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T17:02:29.515853+0000","last_undegraded":"2023-10-18T17:02:29.515853+0000","last_fullsized":"2023-10-18T17:02:29.515853+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1359,"ondisk_log_size":1359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":151,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451303+0000","last_change":"2023-10-18T15:59:17.900205+0000","last_active":"2023-10-18T17:02:28.451303+0000","last_peered":"2023-10-18T17:02:28.451303+0000","last_clean":"2023-10-18T17:02:28.451303+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T17:02:28.451303+0000","last_undegraded":"2023-10-18T17:02:28.451303+0000","last_fullsized":"2023-10-18T17:02:28.451303+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"151'983","reported_seq":1818,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:29.515855+0000","last_change":"2023-10-18T16:03:52.115383+0000","last_active":"2023-10-18T17:02:29.515855+0000","last_peered":"2023-10-18T17:02:29.515855+0000","last_clean":"2023-10-18T17:02:29.515855+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T17:02:29.515855+0000","last_undegraded":"2023-10-18T17:02:29.515855+0000","last_fullsized":"2023-10-18T17:02:29.515855+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":983,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1993,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452064+0000","last_change":"2023-10-18T16:03:52.125315+0000","last_active":"2023-10-18T17:02:28.452064+0000","last_peered":"2023-10-18T17:02:28.452064+0000","last_clean":"2023-10-18T17:02:28.452064+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T17:02:28.452064+0000","last_undegraded":"2023-10-18T17:02:28.452064+0000","last_fullsized":"2023-10-18T17:02:28.452064+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8455,"ondisk_log_size":8455,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740569,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.224,"5min":0.23400000000000001,"15min":0.22900000000000001},"min":{"1min":0.14299999999999999,"5min":0.14299999999999999,"15min":0.14299999999999999},"max":{"1min":0.41199999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.26600000000000001,"15min":0.26200000000000001},"min":{"1min":0.21299999999999999,"5min":0.16,"15min":0.16},"max":{"1min":0.44700000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.28599999999999998}]},{"osd":1,"last update":"Wed Oct 18 19:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.23799999999999999,"15min":0.23699999999999999},"min":{"1min":0.182,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.38200000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.245,"5min":0.25,"15min":0.245},"min":{"1min":0.19400000000000001,"5min":0.16900000000000001,"15min":0.16400000000000001},"max":{"1min":0.42099999999999999,"5min":0.48199999999999998,"15min":0.48599999999999999},"last":0.25700000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740570,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.20599999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"min":{"1min":0.16,"5min":0.153,"15min":0.14099999999999999},"max":{"1min":0.34799999999999998,"5min":0.42499999999999999,"15min":0.436},"last":0.246},{"interface":"front","average":{"1min":0.224,"5min":0.22600000000000001,"15min":0.22800000000000001},"min":{"1min":0.17599999999999999,"5min":0.17299999999999999,"15min":0.16800000000000001},"max":{"1min":0.28599999999999998,"5min":0.433,"15min":0.433},"last":0.27600000000000002}]},{"osd":2,"last update":"Wed Oct 18 19:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.217,"5min":0.22500000000000001,"15min":0.22600000000000001},"min":{"1min":0.184,"5min":0.16700000000000001,"15min":0.154},"max":{"1min":0.29799999999999999,"5min":0.44700000000000001,"15min":0.47599999999999998},"last":0.224},{"interface":"front","average":{"1min":0.246,"5min":0.245,"15min":0.245},"min":{"1min":0.215,"5min":0.17599999999999999,"15min":0.17299999999999999},"max":{"1min":0.311,"5min":0.45800000000000002,"15min":0.45800000000000002},"last":0.28699999999999998}]}]},{"osd":1,"up_from":8,"seq":34359740569,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.224,"5min":0.21099999999999999,"15min":0.20999999999999999},"min":{"1min":0.152,"5min":0.13800000000000001,"15min":0.13800000000000001},"max":{"1min":0.41099999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"last":0.17199999999999999},{"interface":"front","average":{"1min":0.25,"5min":0.24099999999999999,"15min":0.23999999999999999},"min":{"1min":0.18099999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.44,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.21099999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.25900000000000001,"5min":0.23599999999999999,"15min":0.23100000000000001},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.16300000000000001},"max":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"last":0.20300000000000001},{"interface":"front","average":{"1min":0.27100000000000002,"5min":0.26100000000000001,"15min":0.255},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.17000000000000001},"max":{"1min":0.46300000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"last":0.245}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:02:22.488 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-18T19:02:22.489 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:02:22.529 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:22.502+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:22.818 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:02:22.819 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:02:22.831 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5645,"stamp":"2023-10-18T19:02:22.491447+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25628,"ondisk_log_size":25628,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001589"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"151'1359","reported_seq":2175,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:29.515853+0000","last_change":"2023-10-18T16:03:52.115363+0000","last_active":"2023-10-18T17:02:29.515853+0000","last_peered":"2023-10-18T17:02:29.515853+0000","last_clean":"2023-10-18T17:02:29.515853+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T17:02:29.515853+0000","last_undegraded":"2023-10-18T17:02:29.515853+0000","last_fullsized":"2023-10-18T17:02:29.515853+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1359,"ondisk_log_size":1359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":151,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451303+0000","last_change":"2023-10-18T15:59:17.900205+0000","last_active":"2023-10-18T17:02:28.451303+0000","last_peered":"2023-10-18T17:02:28.451303+0000","last_clean":"2023-10-18T17:02:28.451303+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T17:02:28.451303+0000","last_undegraded":"2023-10-18T17:02:28.451303+0000","last_fullsized":"2023-10-18T17:02:28.451303+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"151'983","reported_seq":1818,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:29.515855+0000","last_change":"2023-10-18T16:03:52.115383+0000","last_active":"2023-10-18T17:02:29.515855+0000","last_peered":"2023-10-18T17:02:29.515855+0000","last_clean":"2023-10-18T17:02:29.515855+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T17:02:29.515855+0000","last_undegraded":"2023-10-18T17:02:29.515855+0000","last_fullsized":"2023-10-18T17:02:29.515855+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":983,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1993,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452064+0000","last_change":"2023-10-18T16:03:52.125315+0000","last_active":"2023-10-18T17:02:28.452064+0000","last_peered":"2023-10-18T17:02:28.452064+0000","last_clean":"2023-10-18T17:02:28.452064+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T17:02:28.452064+0000","last_undegraded":"2023-10-18T17:02:28.452064+0000","last_fullsized":"2023-10-18T17:02:28.452064+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8455,"ondisk_log_size":8455,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740570,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.22,"5min":0.22900000000000001,"15min":0.22800000000000001},"min":{"1min":0.161,"5min":0.14299999999999999,"15min":0.14299999999999999},"max":{"1min":0.27600000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.192},{"interface":"front","average":{"1min":0.23699999999999999,"5min":0.25900000000000001,"15min":0.26000000000000001},"min":{"1min":0.16600000000000001,"5min":0.16,"15min":0.16},"max":{"1min":0.28599999999999998,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.26500000000000001}]},{"osd":1,"last update":"Wed Oct 18 19:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.218,"5min":0.23300000000000001,"15min":0.23499999999999999},"min":{"1min":0.186,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.249,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.223},{"interface":"front","average":{"1min":0.222,"5min":0.24299999999999999,"15min":0.24399999999999999},"min":{"1min":0.182,"5min":0.16900000000000001,"15min":0.16400000000000001},"max":{"1min":0.27200000000000002,"5min":0.48199999999999998,"15min":0.48599999999999999},"last":0.254}]}]},{"osd":0,"up_from":8,"seq":34359740571,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.20599999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"min":{"1min":0.16,"5min":0.153,"15min":0.14099999999999999},"max":{"1min":0.34799999999999998,"5min":0.42499999999999999,"15min":0.436},"last":0.247},{"interface":"front","average":{"1min":0.224,"5min":0.22600000000000001,"15min":0.22800000000000001},"min":{"1min":0.17599999999999999,"5min":0.17299999999999999,"15min":0.16800000000000001},"max":{"1min":0.28599999999999998,"5min":0.433,"15min":0.433},"last":0.26100000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.217,"5min":0.22500000000000001,"15min":0.22600000000000001},"min":{"1min":0.184,"5min":0.16700000000000001,"15min":0.154},"max":{"1min":0.29799999999999999,"5min":0.44700000000000001,"15min":0.47599999999999998},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.246,"5min":0.245,"15min":0.245},"min":{"1min":0.215,"5min":0.17599999999999999,"15min":0.17299999999999999},"max":{"1min":0.311,"5min":0.45800000000000002,"15min":0.45800000000000002},"last":0.27400000000000002}]}]},{"osd":1,"up_from":8,"seq":34359740570,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.224,"5min":0.21099999999999999,"15min":0.20999999999999999},"min":{"1min":0.152,"5min":0.13800000000000001,"15min":0.13800000000000001},"max":{"1min":0.41099999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"last":0.246},{"interface":"front","average":{"1min":0.25,"5min":0.24099999999999999,"15min":0.23999999999999999},"min":{"1min":0.18099999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.44,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.215}]},{"osd":2,"last update":"Wed Oct 18 19:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.25900000000000001,"5min":0.23599999999999999,"15min":0.23100000000000001},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.16300000000000001},"max":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.27100000000000002,"5min":0.26100000000000001,"15min":0.255},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.17000000000000001},"max":{"1min":0.46300000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"last":0.27200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:02:22.832 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T19:02:23.140 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:02:23.140 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":154,"fsid":"cb63dacd-a4a9-467a-b30e-f9493a2182c3","created":"2023-10-18T15:59:07.909789+0000","modified":"2023-10-18T17:02:33.449830+0000","last_up_change":"2023-10-18T15:59:15.884655+0000","last_in_change":"2023-10-18T15:59:08.812513+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-10-18T15:59:15.923203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T15:59:16.676775+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"15","last_force_op_resend_prenautilus":"15","last_force_op_resend_preluminous":"15","auid":0,"snap_mode":"selfmanaged","snap_seq":91,"snap_epoch":150,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cache","create_time":"2023-10-18T15:59:24.096546+0000","flags":8201,"flags_names":"hashpspool,incomplete_clones,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"15","last_force_op_resend_prenautilus":"15","last_force_op_resend_preluminous":"15","auid":0,"snap_mode":"selfmanaged","snap_seq":91,"snap_epoch":150,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":2,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"f646c787-fe92-4e2d-abce-6b81d0e6e284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":87291},{"type":"v1","addr":"172.21.15.82:6817","nonce":87291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":87291},{"type":"v1","addr":"172.21.15.82:6819","nonce":87291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":87291},{"type":"v1","addr":"172.21.15.82:6823","nonce":87291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":87291},{"type":"v1","addr":"172.21.15.82:6821","nonce":87291}]},"public_addr":"172.21.15.82:6817/87291","cluster_addr":"172.21.15.82:6819/87291","heartbeat_back_addr":"172.21.15.82:6823/87291","heartbeat_front_addr":"172.21.15.82:6821/87291","state":["exists","up"]},{"osd":1,"uuid":"46810abb-ff37-4aeb-b272-7610274b1957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":87261},{"type":"v1","addr":"172.21.15.82:6809","nonce":87261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":87261},{"type":"v1","addr":"172.21.15.82:6811","nonce":87261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":87261},{"type":"v1","addr":"172.21.15.82:6815","nonce":87261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":87261},{"type":"v1","addr":"172.21.15.82:6813","nonce":87261}]},"public_addr":"172.21.15.82:6809/87261","cluster_addr":"172.21.15.82:6811/87261","heartbeat_back_addr":"172.21.15.82:6815/87261","heartbeat_front_addr":"172.21.15.82:6813/87261","state":["exists","up"]},{"osd":2,"uuid":"a5410593-60ce-4cb3-901e-e96645dc2c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":87262},{"type":"v1","addr":"172.21.15.82:6801","nonce":87262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":87262},{"type":"v1","addr":"172.21.15.82:6803","nonce":87262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":87262},{"type":"v1","addr":"172.21.15.82:6807","nonce":87262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":87262},{"type":"v1","addr":"172.21.15.82:6805","nonce":87262}]},"public_addr":"172.21.15.82:6801/87262","cluster_addr":"172.21.15.82:6803/87262","heartbeat_back_addr":"172.21.15.82:6807/87262","heartbeat_front_addr":"172.21.15.82:6805/87262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T15:59:14.031532+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T15:59:13.937892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T15:59:13.949251+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":4,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]},{"pool":5,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]},{"pool":7,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T19:02:23.150 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:23.148+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:23.519 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:23.516+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:24.140 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:24.137+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:24.155 INFO:tasks.ceph:Scrubbing osd.0 2023-10-18T19:02:24.155 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-18T19:02:24.537 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:24.534+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:25.108 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:25.106+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:25.518 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:25.516+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:26.108 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:26.106+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:26.538 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:26.536+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:27.084 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:27.082+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:27.511 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:27.510+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:28.045 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:28.043+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:28.483 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:28.481+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:29.069 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:29.066+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:29.501 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:29.498+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:30.051 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:30.050+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:30.467 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:30.465+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:31.079 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:31.077+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:31.491 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:31.490+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:32.107 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:32.105+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:32.462 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:32.460+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:33.103 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:33.101+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:33.499 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:33.496+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:34.148 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:34.146+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:34.509 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:34.507+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:35.103 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:35.101+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:35.469 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:35.468+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:36.088 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:36.086+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:36.469 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:36.468+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:37.086 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:37.084+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:37.430 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:37.428+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:38.110 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:38.109+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:38.427 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:38.425+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:39.105 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:39.104+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:39.415 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:39.413+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:40.071 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:40.069+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:40.447 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:40.445+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:41.090 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:41.089+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:41.400 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:41.398+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:42.083 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:42.080+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:42.450 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:42.448+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:43.075 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:43.072+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:43.474 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:43.471+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:44.110 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:44.107+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:44.427 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:44.424+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:45.104 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:45.101+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:45.411 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:45.408+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:46.097 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:46.094+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:46.363 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:46.360+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:47.126 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:47.123+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:47.395 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:47.392+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:48.096 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:48.093+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:48.436 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:48.433+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:49.072 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:49.069+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:49.472 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:49.469+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:50.066 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:50.063+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:50.446 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:50.443+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:51.096 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:51.093+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:51.451 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:51.448+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:52.057 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:52.055+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:52.420 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:52.418+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:53.035 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:53.032+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:53.402 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:53.400+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:54.083 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:54.080+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:54.430 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:54.427+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:55.044 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:55.041+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:55.400 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:55.398+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:56.065 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:56.062+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:56.427 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:56.424+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:57.064 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:57.061+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:57.394 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:57.391+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:58.087 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:58.084+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:58.362 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:58.359+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:59.062 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:02:59.059+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:02:59.363 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:02:59.360+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:00.109 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:00.106+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:00.377 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:00.375+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:01.112 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:01.109+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:01.333 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:01.330+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:02.136 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:02.133+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:02.319 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:02.316+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:03.159 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:03.157+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:03.307 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:03.305+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:04.183 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:04.180+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:04.331 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:04.329+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:05.161 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:05.159+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:05.300 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:05.297+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:06.196 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:06.193+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:06.273 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:06.271+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:07.179 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:07.176+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:07.235 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:07.232+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:08.167 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:08.164+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:08.254 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:08.251+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:09.178 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:09.176+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:09.258 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:09.255+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:10.158 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:10.155+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:10.265 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:10.261+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:11.111 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:11.109+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:11.288 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:11.285+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:12.086 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:12.083+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:12.335 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:12.332+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:13.064 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:13.062+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:13.352 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:13.350+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:14.095 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:14.092+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:14.348 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:14.345+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:15.098 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:15.096+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:15.362 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:15.360+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:16.113 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:16.111+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:16.387 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:16.386+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:17.160 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:17.158+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:17.409 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:17.408+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:18.118 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:18.116+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:18.423 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:18.422+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:19.092 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:19.090+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:19.411 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:19.410+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:20.136 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:20.134+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:20.460 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:20.458+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:21.150 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:21.148+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:21.472 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:21.470+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:22.106 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:22.104+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:22.471 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:22.470+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:23.079 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:23.077+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:23.459 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:23.458+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:24.045 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:24.044+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:24.443 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:24.442+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:25.066 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:25.065+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:25.394 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:25.392+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:26.021 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:26.020+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:26.348 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:26.346+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:27.063 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:27.061+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:27.316 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:27.314+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:28.039 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:28.037+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:28.280 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:28.279+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:29.029 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:29.027+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:29.326 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:29.325+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:30.076 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:30.074+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:30.302 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:30.300+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:31.078 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:31.076+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:31.306 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:31.304+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:32.088 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:32.086+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:32.309 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:32.306+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:33.080 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:33.077+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:33.348 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:33.345+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:34.032 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:34.029+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:34.383 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:34.381+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:34.998 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:34.995+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:35.335 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:35.332+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:35.958 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:35.955+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:36.337 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:36.334+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:36.966 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:36.963+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:37.357 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:37.354+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:37.984 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:37.981+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:38.406 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:38.403+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:39.008 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:39.006+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:39.401 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:39.398+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:40.013 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:40.010+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:40.437 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:40.435+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:41.042 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:41.039+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:41.458 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:41.455+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:42.001 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:41.999+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:42.424 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:42.421+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:42.963 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:42.961+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:43.471 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:43.468+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:43.931 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:43.928+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:44.498 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:44.495+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:44.952 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:44.950+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:45.539 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:45.537+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:45.905 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:45.902+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:46.514 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:46.511+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:46.888 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:46.885+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:47.526 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:47.523+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:47.918 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:47.915+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:48.509 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:48.507+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:48.877 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:48.874+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:49.550 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:49.547+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:49.902 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:49.899+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:50.528 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:50.525+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:50.949 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:50.946+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:51.551 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:51.548+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:51.934 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:51.932+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:52.546 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:52.543+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:52.922 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:52.919+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:53.544 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:53.541+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:53.921 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:53.918+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:54.517 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:54.515+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:54.909 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:54.907+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:55.505 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:55.502+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:55.871 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:55.868+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:56.499 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:56.496+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:56.853 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:56.850+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:57.544 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:57.541+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:57.804 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:57.801+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:58.559 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:58.556+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:58.784 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:58.781+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:59.520 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:03:59.517+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:03:59.825 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:03:59.822+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:00.524 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:00.522+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:00.872 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:00.869+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:01.533 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:01.531+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:01.868 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:01.865+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:02.559 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:02.556+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:02.900 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:02.897+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:03.603 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:03.601+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:03.942 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:03.939+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:04.610 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:04.607+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:04.961 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:04.959+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:05.622 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:05.620+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:05.960 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:05.958+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:06.576 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:06.574+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:06.917 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:06.915+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:07.540 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:07.538+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:07.914 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:07.912+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:08.513 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:08.511+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:08.919 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:08.918+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:09.520 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:09.518+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:09.918 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:09.916+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:10.505 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:10.503+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:10.951 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:10.949+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:11.516 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:11.514+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:11.919 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:11.917+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:12.519 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:12.518+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:12.960 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:12.958+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:13.495 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:13.493+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:13.944 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:13.943+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:14.523 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:14.521+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:14.960 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:14.958+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:15.501 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:15.499+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:15.963 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:15.962+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:16.475 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:16.473+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:16.995 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:16.993+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:17.506 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:17.504+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:17.981 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:17.980+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:18.529 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:18.527+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:18.941 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:18.939+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:19.496 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:19.495+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:19.964 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:19.962+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:20.490 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:20.488+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:20.928 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:20.926+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:21.488 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:21.487+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:21.967 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:21.966+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:22.446 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:22.443+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:22.923 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:22.920+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:23.460 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:23.457+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:23.924 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:23.921+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:24.184 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-10-18T19:04:24.185 INFO:tasks.ceph:Scrubbing osd.1 2023-10-18T19:04:24.186 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-18T19:04:24.466 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:24.463+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:24.945 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:24.942+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:25.471 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:25.468+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:25.989 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:25.987+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:26.473 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:26.470+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:27.022 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:27.019+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:27.434 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:27.431+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:28.068 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:28.065+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:28.448 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:28.445+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:29.085 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:29.082+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:29.446 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:29.444+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:30.072 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:30.070+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:30.429 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:30.426+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:31.092 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:31.090+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:31.402 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:31.399+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:32.114 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:32.112+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:32.398 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:32.395+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:33.134 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:33.131+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:33.389 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:33.386+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:34.152 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:34.149+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:34.387 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:34.385+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:35.187 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:35.184+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:35.366 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:35.363+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:36.146 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:36.143+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:36.387 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:36.384+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:37.130 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:37.127+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:37.387 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:37.384+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:38.146 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:38.143+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:38.434 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:38.431+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:39.142 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:39.139+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:39.395 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:39.392+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:40.122 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:40.119+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:40.427 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:40.424+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:41.089 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:41.086+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:41.378 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:41.375+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:42.123 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:42.120+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:42.425 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:42.422+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:43.167 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:43.165+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:43.394 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:43.391+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:44.125 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:44.123+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:44.396 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:44.393+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:45.162 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:45.159+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:45.406 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:45.403+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:46.154 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:46.151+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:46.368 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:46.365+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:47.198 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:47.195+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:47.361 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:47.358+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:48.213 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:48.210+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:48.337 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:48.334+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:49.229 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:49.226+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:49.308 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:49.305+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:50.189 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:50.186+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:50.308 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:50.305+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:51.188 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:51.185+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:51.332 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:51.329+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:52.151 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:52.148+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:52.316 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:52.314+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:53.170 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:53.167+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:53.275 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:53.272+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:54.179 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:54.176+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:54.265 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:54.262+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:55.158 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:55.156+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:55.258 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:55.256+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:56.150 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:56.148+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:56.273 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:56.271+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:57.197 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:57.195+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:57.234 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:57.232+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:58.212 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:58.210+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:58.246 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:58.244+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:59.211 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:04:59.209+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:04:59.254 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:04:59.252+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:00.181 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:00.179+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:00.290 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:00.289+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:01.194 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:01.193+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:01.244 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:01.242+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:02.160 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:02.158+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:02.289 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:02.287+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:03.159 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:03.157+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:03.265 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:03.263+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:04.163 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:04.161+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:04.297 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:04.295+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:05.159 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:05.157+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:05.254 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:05.253+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:06.198 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:06.197+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:06.207 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:06.206+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:07.157 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:07.155+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:07.197 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:07.196+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:08.186 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:08.184+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:08.201 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:08.199+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:09.210 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:09.208+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:09.225 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:09.224+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:10.201 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:10.199+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:10.214 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:10.212+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:11.217 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:11.215+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:11.233 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:11.231+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:12.209 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:12.206+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:12.261 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:12.259+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:13.172 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:13.169+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:13.213 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:13.211+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:14.174 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:14.172+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:14.209 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:14.206+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:15.187 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:15.184+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:15.193 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:15.190+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:16.197 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:16.195+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:16.215 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:16.212+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:17.179 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:17.176+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:17.196 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:17.193+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:18.194 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:18.191+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:18.224 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:18.221+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:19.160 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:19.157+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:19.262 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:19.259+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:20.194 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:20.191+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:20.312 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:20.309+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:21.233 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:21.230+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:21.297 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:21.294+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:22.244 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:22.241+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:22.346 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:22.344+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:23.221 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:23.218+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:23.327 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:23.324+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:24.227 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:24.224+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:24.375 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:24.372+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:25.227 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:25.224+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:25.342 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:25.339+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:26.231 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:26.228+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:26.328 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:26.325+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:27.227 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:27.225+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:27.312 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:27.309+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:28.244 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:28.241+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:28.320 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:28.318+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:29.293 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:29.290+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:29.353 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:29.350+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:30.258 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:30.255+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:30.341 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:30.338+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:31.265 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:31.262+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:31.330 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:31.327+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:32.244 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:32.242+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:32.312 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:32.309+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:33.203 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:33.200+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:33.297 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:33.294+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:34.202 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:34.199+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:34.306 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:34.303+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:35.230 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:35.216+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:35.341 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:35.338+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:36.204 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:36.201+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:36.361 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:36.359+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:37.254 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:37.251+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:37.384 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:37.381+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:38.264 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:38.261+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:38.406 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:38.403+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:39.286 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:39.283+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:39.381 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:39.378+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:40.331 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:40.328+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:40.431 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:40.428+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:41.377 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:41.374+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:41.416 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:41.413+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:42.370 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:42.367+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:42.424 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:42.421+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:43.329 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:43.326+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:43.455 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:43.452+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:44.348 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:44.345+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:44.410 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:44.408+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:45.397 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:45.395+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:45.447 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:45.445+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:46.355 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:46.353+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:46.448 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:46.446+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:47.382 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:47.380+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:47.417 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:47.415+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:48.388 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:48.386+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:48.427 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:48.425+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:49.343 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:49.341+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:49.449 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:49.448+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:50.379 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:50.377+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:50.499 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:50.497+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:51.382 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:51.380+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:51.461 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:51.459+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:52.430 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:52.428+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:52.483 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:52.481+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:53.426 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:53.424+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:53.520 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:53.519+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:54.426 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:54.424+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:54.510 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:54.508+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:55.387 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:55.385+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:55.491 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:55.489+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:56.354 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:56.352+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:56.533 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:56.531+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:57.379 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:57.377+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:57.539 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:57.537+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:58.386 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:58.384+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:58.521 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:58.519+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:59.430 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:05:59.428+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:05:59.522 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:05:59.521+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:00.465 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:00.463+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:00.558 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:00.556+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:01.417 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:01.414+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:01.604 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:01.603+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:02.428 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:02.425+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:02.582 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:02.579+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:03.396 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:03.393+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:03.563 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:03.561+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:04.446 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:04.444+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:04.597 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:04.594+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:05.482 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:05.479+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:05.625 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:05.622+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:06.499 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:06.496+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:06.580 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:06.577+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:07.494 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:07.491+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:07.535 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:07.532+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:08.490 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:08.486+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:08.537 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:08.534+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:09.512 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:09.509+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:09.575 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:09.572+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:10.561 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:10.558+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:10.595 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:10.593+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:11.539 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:11.536+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:11.589 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:11.586+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:12.566 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:12.563+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:12.634 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:12.631+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:13.536 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:13.533+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:13.607 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:13.604+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:14.563 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:14.560+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:14.593 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:14.590+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:15.602 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:15.599+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:15.604 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:15.602+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:16.595 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:16.592+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:16.618 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:16.615+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:17.561 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:17.558+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:17.608 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:17.605+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:18.563 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:18.560+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:18.584 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:18.581+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:19.557 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:19.554+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:19.624 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:19.621+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:20.538 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:20.535+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:20.628 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:20.625+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:21.525 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:21.522+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:21.598 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:21.596+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:22.484 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:22.481+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:22.647 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:22.644+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:23.456 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:23.454+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:23.645 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:23.642+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:24.215 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-10-18T19:06:24.217 INFO:tasks.ceph:Scrubbing osd.2 2023-10-18T19:06:24.217 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-18T19:06:24.395 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-10-18T19:06:24.396 INFO:teuthology.orchestra.run.smithi082.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-18T19:06:24.396 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-10-18T19:06:24.408 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-18T19:06:24.463 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:24.460+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:24.658 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:24.655+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:24.708 INFO:teuthology.orchestra.run.smithi082.stderr:instructed osd(s) 2 to deep-scrub 2023-10-18T19:06:24.720 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:06:25.012 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:06:25.014 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:06:25.026 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5766,"stamp":"2023-10-18T19:06:24.525658+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25628,"ondisk_log_size":25628,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001745"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"151'1359","reported_seq":2175,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:29.515853+0000","last_change":"2023-10-18T16:03:52.115363+0000","last_active":"2023-10-18T17:02:29.515853+0000","last_peered":"2023-10-18T17:02:29.515853+0000","last_clean":"2023-10-18T17:02:29.515853+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T17:02:29.515853+0000","last_undegraded":"2023-10-18T17:02:29.515853+0000","last_fullsized":"2023-10-18T17:02:29.515853+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1359,"ondisk_log_size":1359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":151,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451303+0000","last_change":"2023-10-18T15:59:17.900205+0000","last_active":"2023-10-18T17:02:28.451303+0000","last_peered":"2023-10-18T17:02:28.451303+0000","last_clean":"2023-10-18T17:02:28.451303+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T17:02:28.451303+0000","last_undegraded":"2023-10-18T17:02:28.451303+0000","last_fullsized":"2023-10-18T17:02:28.451303+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"151'983","reported_seq":1818,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:29.515855+0000","last_change":"2023-10-18T16:03:52.115383+0000","last_active":"2023-10-18T17:02:29.515855+0000","last_peered":"2023-10-18T17:02:29.515855+0000","last_clean":"2023-10-18T17:02:29.515855+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T17:02:29.515855+0000","last_undegraded":"2023-10-18T17:02:29.515855+0000","last_fullsized":"2023-10-18T17:02:29.515855+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":983,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1993,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452064+0000","last_change":"2023-10-18T16:03:52.125315+0000","last_active":"2023-10-18T17:02:28.452064+0000","last_peered":"2023-10-18T17:02:28.452064+0000","last_clean":"2023-10-18T17:02:28.452064+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T17:02:28.452064+0000","last_undegraded":"2023-10-18T17:02:28.452064+0000","last_fullsized":"2023-10-18T17:02:28.452064+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8455,"ondisk_log_size":8455,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740618,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.214,"5min":0.22,"15min":0.23000000000000001},"min":{"1min":0.159,"5min":0.14299999999999999,"15min":0.14299999999999999},"max":{"1min":0.377,"5min":0.41199999999999998,"15min":0.47999999999999998},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.255,"15min":0.26300000000000001},"min":{"1min":0.216,"5min":0.16600000000000001,"15min":0.16},"max":{"1min":0.433,"5min":0.44700000000000001,"15min":0.51000000000000001},"last":0.311}]},{"osd":1,"last update":"Wed Oct 18 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.22900000000000001,"15min":0.23599999999999999},"min":{"1min":0.193,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.40100000000000002,"5min":0.40200000000000002,"15min":0.48899999999999999},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.24399999999999999,"5min":0.23499999999999999,"15min":0.24399999999999999},"min":{"1min":0.187,"5min":0.17899999999999999,"15min":0.16400000000000001},"max":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.48599999999999999},"last":0.29399999999999998}]}]},{"osd":0,"up_from":8,"seq":34359740619,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:05:17 2023","interfaces":[{"interface":"back","average":{"1min":0.19500000000000001,"5min":0.19600000000000001,"15min":0.21099999999999999},"min":{"1min":0.13200000000000001,"5min":0.11799999999999999,"15min":0.11799999999999999},"max":{"1min":0.34200000000000003,"5min":0.34799999999999998,"15min":0.436},"last":0.17999999999999999},{"interface":"front","average":{"1min":0.22900000000000001,"5min":0.222,"15min":0.23000000000000001},"min":{"1min":0.17399999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.374,"5min":0.374,"15min":0.433},"last":0.20799999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:05:17 2023","interfaces":[{"interface":"back","average":{"1min":0.23499999999999999,"5min":0.217,"15min":0.22800000000000001},"min":{"1min":0.17000000000000001,"5min":0.16600000000000001,"15min":0.158},"max":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.47599999999999998},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.23999999999999999,"5min":0.23599999999999999,"15min":0.245},"min":{"1min":0.17199999999999999,"5min":0.14399999999999999,"15min":0.14399999999999999},"max":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.45800000000000002},"last":0.24299999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740619,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:06:07 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.214,"15min":0.21299999999999999},"min":{"1min":0.16800000000000001,"5min":0.152,"15min":0.13800000000000001},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.17799999999999999},{"interface":"front","average":{"1min":0.27200000000000002,"5min":0.245,"15min":0.24299999999999999},"min":{"1min":0.20300000000000001,"5min":0.18099999999999999,"15min":0.17299999999999999},"max":{"1min":0.40400000000000003,"5min":0.44,"15min":0.45300000000000001},"last":0.20899999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:06:07 2023","interfaces":[{"interface":"back","average":{"1min":0.25600000000000001,"5min":0.23799999999999999,"15min":0.23300000000000001},"min":{"1min":0.18099999999999999,"5min":0.16900000000000001,"15min":0.16300000000000001},"max":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"last":0.191},{"interface":"front","average":{"1min":0.28699999999999998,"5min":0.26300000000000001,"15min":0.25900000000000001},"min":{"1min":0.20399999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.22800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:06:25.030 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:06:25.031 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:06:25.031 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:06:25.032 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:06:25.032 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:06:25.032 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:06:25.033 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:06:25.033 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:06:25.034 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:06:25.034 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:06:25.034 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:06:25.035 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:06:25.035 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:06:25.035 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:06:25.492 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:25.489+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:25.619 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:25.616+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:26.542 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:26.539+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:26.665 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:26.662+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:27.576 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:27.574+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:27.651 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:27.648+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:28.606 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:28.603+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:28.693 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:28.690+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:29.590 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:29.587+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:29.678 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:29.675+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:30.621 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:30.617+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:30.667 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:30.664+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:31.605 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:31.602+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:31.664 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:31.662+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:32.592 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:32.590+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:32.711 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:32.708+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:33.587 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:33.584+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:33.714 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:33.711+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:34.597 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:34.594+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:34.674 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:34.672+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:35.640 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:35.638+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:35.657 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:35.655+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:36.609 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:36.607+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:36.690 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:36.688+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:37.575 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:37.573+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:37.651 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:37.649+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:38.538 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:38.536+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:38.641 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:38.639+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:39.537 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:39.535+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:39.630 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:39.628+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:40.502 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:40.500+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:40.585 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:40.583+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:41.545 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:41.543+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:41.571 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:41.569+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:42.534 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:42.532+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:42.586 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:42.584+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:43.561 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:43.559+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:43.573 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:43.571+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:44.547 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:44.545+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:44.598 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:44.596+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:45.036 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:06:45.341 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:06:45.342 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:06:45.353 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5776,"stamp":"2023-10-18T19:06:44.528664+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001870"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":151,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451303+0000","last_change":"2023-10-18T15:59:17.900205+0000","last_active":"2023-10-18T17:02:28.451303+0000","last_peered":"2023-10-18T17:02:28.451303+0000","last_clean":"2023-10-18T17:02:28.451303+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T17:02:28.451303+0000","last_undegraded":"2023-10-18T17:02:28.451303+0000","last_fullsized":"2023-10-18T17:02:28.451303+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1993,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452064+0000","last_change":"2023-10-18T16:03:52.125315+0000","last_active":"2023-10-18T17:02:28.452064+0000","last_peered":"2023-10-18T17:02:28.452064+0000","last_clean":"2023-10-18T17:02:28.452064+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T17:02:28.452064+0000","last_undegraded":"2023-10-18T17:02:28.452064+0000","last_fullsized":"2023-10-18T17:02:28.452064+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740622,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:06:29 2023","interfaces":[{"interface":"back","average":{"1min":0.23100000000000001,"5min":0.221,"15min":0.23000000000000001},"min":{"1min":0.14999999999999999,"5min":0.14999999999999999,"15min":0.14299999999999999},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.47999999999999998},"last":0.17000000000000001},{"interface":"front","average":{"1min":0.27600000000000002,"5min":0.25700000000000001,"15min":0.26400000000000001},"min":{"1min":0.20200000000000001,"5min":0.16600000000000001,"15min":0.16},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.51000000000000001},"last":0.24399999999999999}]},{"osd":1,"last update":"Wed Oct 18 19:06:29 2023","interfaces":[{"interface":"back","average":{"1min":0.245,"5min":0.23100000000000001,"15min":0.23599999999999999},"min":{"1min":0.17799999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.20300000000000001},{"interface":"front","average":{"1min":0.255,"5min":0.23699999999999999,"15min":0.245},"min":{"1min":0.189,"5min":0.17899999999999999,"15min":0.16400000000000001},"max":{"1min":0.436,"5min":0.436,"15min":0.48599999999999999},"last":0.216}]}]},{"osd":0,"up_from":8,"seq":34359740623,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.20000000000000001,"5min":0.19800000000000001,"15min":0.20999999999999999},"min":{"1min":0.159,"5min":0.11799999999999999,"15min":0.11799999999999999},"max":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.436},"last":0.16700000000000001},{"interface":"front","average":{"1min":0.22700000000000001,"5min":0.22500000000000001,"15min":0.23000000000000001},"min":{"1min":0.17599999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.33400000000000002,"5min":0.374,"15min":0.433},"last":0.187}]},{"osd":2,"last update":"Wed Oct 18 19:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.222,"15min":0.22900000000000001},"min":{"1min":0.184,"5min":0.16600000000000001,"15min":0.158},"max":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.47599999999999998},"last":0.20100000000000001},{"interface":"front","average":{"1min":0.246,"5min":0.23899999999999999,"15min":0.246},"min":{"1min":0.20000000000000001,"5min":0.14399999999999999,"15min":0.14399999999999999},"max":{"1min":0.372,"5min":0.39600000000000002,"15min":0.45800000000000002},"last":0.22900000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740623,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:06:07 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.214,"15min":0.21299999999999999},"min":{"1min":0.16800000000000001,"5min":0.152,"15min":0.13800000000000001},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.17499999999999999},{"interface":"front","average":{"1min":0.27200000000000002,"5min":0.245,"15min":0.24299999999999999},"min":{"1min":0.20300000000000001,"5min":0.18099999999999999,"15min":0.17299999999999999},"max":{"1min":0.40400000000000003,"5min":0.44,"15min":0.45300000000000001},"last":0.20100000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:06:07 2023","interfaces":[{"interface":"back","average":{"1min":0.25600000000000001,"5min":0.23799999999999999,"15min":0.23300000000000001},"min":{"1min":0.18099999999999999,"5min":0.16900000000000001,"15min":0.16300000000000001},"max":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"last":0.219},{"interface":"front","average":{"1min":0.28699999999999998,"5min":0.26300000000000001,"15min":0.25900000000000001},"min":{"1min":0.20399999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.22800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:06:45.355 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:06:45.355 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:06:45.356 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:06:45.356 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:06:45.357 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:06:45.357 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:06:45.357 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:06:45.358 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:06:45.358 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:06:45.358 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:06:45.359 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:06:45.361 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:06:45.596 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:45.594+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:45.632 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:45.630+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:46.584 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:46.582+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:46.659 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:46.657+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:47.586 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:47.584+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:47.656 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:47.654+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:48.598 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:48.596+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:48.607 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:48.606+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:49.593 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:49.591+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:49.616 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:49.615+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:50.604 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:50.601+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:50.608 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:50.606+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:51.572 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:51.570+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:51.619 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:51.618+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:52.524 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:52.521+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:52.666 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:52.663+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:53.480 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:53.477+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:53.679 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:53.676+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:54.519 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:54.516+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:54.713 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:54.710+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:55.527 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:55.524+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:55.690 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:55.687+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:56.488 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:56.485+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:56.726 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:56.723+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:57.448 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:57.445+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:57.747 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:57.744+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:58.420 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:58.417+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:58.770 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:58.767+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:59.387 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:06:59.384+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:06:59.782 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:06:59.779+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:00.420 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:00.417+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:00.795 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:00.792+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:01.443 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:01.440+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:01.798 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:01.796+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:02.468 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:02.465+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:02.819 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:02.816+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:03.506 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:03.503+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:03.848 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:03.845+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:04.496 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:04.493+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:04.818 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:04.815+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:05.363 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:07:05.479 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:05.476+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:05.668 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:07:05.670 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:07:05.681 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5786,"stamp":"2023-10-18T19:07:04.532051+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002087"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":151,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451303+0000","last_change":"2023-10-18T15:59:17.900205+0000","last_active":"2023-10-18T17:02:28.451303+0000","last_peered":"2023-10-18T17:02:28.451303+0000","last_clean":"2023-10-18T17:02:28.451303+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T17:02:28.451303+0000","last_undegraded":"2023-10-18T17:02:28.451303+0000","last_fullsized":"2023-10-18T17:02:28.451303+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1993,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452064+0000","last_change":"2023-10-18T16:03:52.125315+0000","last_active":"2023-10-18T17:02:28.452064+0000","last_peered":"2023-10-18T17:02:28.452064+0000","last_clean":"2023-10-18T17:02:28.452064+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T17:02:28.452064+0000","last_undegraded":"2023-10-18T17:02:28.452064+0000","last_fullsized":"2023-10-18T17:02:28.452064+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740626,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:06:29 2023","interfaces":[{"interface":"back","average":{"1min":0.23100000000000001,"5min":0.221,"15min":0.23000000000000001},"min":{"1min":0.14999999999999999,"5min":0.14999999999999999,"15min":0.14299999999999999},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.47999999999999998},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.27600000000000002,"5min":0.25700000000000001,"15min":0.26400000000000001},"min":{"1min":0.20200000000000001,"5min":0.16600000000000001,"15min":0.16},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.51000000000000001},"last":0.25900000000000001}]},{"osd":1,"last update":"Wed Oct 18 19:06:29 2023","interfaces":[{"interface":"back","average":{"1min":0.245,"5min":0.23100000000000001,"15min":0.23599999999999999},"min":{"1min":0.17799999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.224},{"interface":"front","average":{"1min":0.255,"5min":0.23699999999999999,"15min":0.245},"min":{"1min":0.189,"5min":0.17899999999999999,"15min":0.16400000000000001},"max":{"1min":0.436,"5min":0.436,"15min":0.48599999999999999},"last":0.248}]}]},{"osd":0,"up_from":8,"seq":34359740627,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.20000000000000001,"5min":0.19800000000000001,"15min":0.20999999999999999},"min":{"1min":0.159,"5min":0.11799999999999999,"15min":0.11799999999999999},"max":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.436},"last":0.18099999999999999},{"interface":"front","average":{"1min":0.22700000000000001,"5min":0.22500000000000001,"15min":0.23000000000000001},"min":{"1min":0.17599999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.33400000000000002,"5min":0.374,"15min":0.433},"last":0.192}]},{"osd":2,"last update":"Wed Oct 18 19:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.222,"15min":0.22900000000000001},"min":{"1min":0.184,"5min":0.16600000000000001,"15min":0.158},"max":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.47599999999999998},"last":0.215},{"interface":"front","average":{"1min":0.246,"5min":0.23899999999999999,"15min":0.246},"min":{"1min":0.20000000000000001,"5min":0.14399999999999999,"15min":0.14399999999999999},"max":{"1min":0.372,"5min":0.39600000000000002,"15min":0.45800000000000002},"last":0.26200000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740627,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:06:07 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.214,"15min":0.21299999999999999},"min":{"1min":0.16800000000000001,"5min":0.152,"15min":0.13800000000000001},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.19600000000000001},{"interface":"front","average":{"1min":0.27200000000000002,"5min":0.245,"15min":0.24299999999999999},"min":{"1min":0.20300000000000001,"5min":0.18099999999999999,"15min":0.17299999999999999},"max":{"1min":0.40400000000000003,"5min":0.44,"15min":0.45300000000000001},"last":0.221}]},{"osd":2,"last update":"Wed Oct 18 19:06:07 2023","interfaces":[{"interface":"back","average":{"1min":0.25600000000000001,"5min":0.23799999999999999,"15min":0.23300000000000001},"min":{"1min":0.18099999999999999,"5min":0.16900000000000001,"15min":0.16300000000000001},"max":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.28699999999999998,"5min":0.26300000000000001,"15min":0.25900000000000001},"min":{"1min":0.20399999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.247}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:07:05.683 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:05.683 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:05.683 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:05.684 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:05.684 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:05.684 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:05.685 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:05.685 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:05.686 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:05.686 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:05.687 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:05.687 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:07:05.851 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:05.849+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:06.524 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:06.521+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:06.806 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:06.803+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:07.480 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:07.477+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:07.790 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:07.787+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:08.486 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:08.483+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:08.757 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:08.754+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:09.534 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:09.531+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:09.727 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:09.724+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:10.563 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:10.560+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:10.717 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:10.714+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:11.558 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:11.555+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:11.768 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:11.765+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:12.580 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:12.577+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:12.746 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:12.743+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:13.555 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:13.552+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:13.787 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:13.784+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:14.521 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:14.519+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:14.764 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:14.761+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:15.477 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:15.475+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:15.811 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:15.808+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:16.527 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:16.524+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:16.821 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:16.818+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:17.515 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:17.512+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:17.870 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:17.867+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:18.562 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:18.559+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:18.863 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:18.860+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:19.540 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:19.537+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:19.887 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:19.884+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:20.581 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:20.578+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:20.846 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:20.843+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:21.586 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:21.583+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:21.824 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:21.821+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:22.614 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:22.611+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:22.829 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:22.826+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:23.572 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:23.569+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:23.844 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:23.841+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:24.565 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:24.562+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:24.840 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:24.838+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:25.521 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:25.519+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:25.688 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:07:25.821 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:25.819+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:25.991 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:07:25.992 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:07:26.003 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5796,"stamp":"2023-10-18T19:07:24.534860+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001757"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":151,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451303+0000","last_change":"2023-10-18T15:59:17.900205+0000","last_active":"2023-10-18T17:02:28.451303+0000","last_peered":"2023-10-18T17:02:28.451303+0000","last_clean":"2023-10-18T17:02:28.451303+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T17:02:28.451303+0000","last_undegraded":"2023-10-18T17:02:28.451303+0000","last_fullsized":"2023-10-18T17:02:28.451303+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1993,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452064+0000","last_change":"2023-10-18T16:03:52.125315+0000","last_active":"2023-10-18T17:02:28.452064+0000","last_peered":"2023-10-18T17:02:28.452064+0000","last_clean":"2023-10-18T17:02:28.452064+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T17:02:28.452064+0000","last_undegraded":"2023-10-18T17:02:28.452064+0000","last_fullsized":"2023-10-18T17:02:28.452064+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740630,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:06:29 2023","interfaces":[{"interface":"back","average":{"1min":0.23100000000000001,"5min":0.221,"15min":0.23000000000000001},"min":{"1min":0.14999999999999999,"5min":0.14999999999999999,"15min":0.14299999999999999},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.47999999999999998},"last":0.16400000000000001},{"interface":"front","average":{"1min":0.27600000000000002,"5min":0.25700000000000001,"15min":0.26400000000000001},"min":{"1min":0.20200000000000001,"5min":0.16600000000000001,"15min":0.16},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.51000000000000001},"last":0.23599999999999999}]},{"osd":1,"last update":"Wed Oct 18 19:06:29 2023","interfaces":[{"interface":"back","average":{"1min":0.245,"5min":0.23100000000000001,"15min":0.23599999999999999},"min":{"1min":0.17799999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.19700000000000001},{"interface":"front","average":{"1min":0.255,"5min":0.23699999999999999,"15min":0.245},"min":{"1min":0.189,"5min":0.17899999999999999,"15min":0.16400000000000001},"max":{"1min":0.436,"5min":0.436,"15min":0.48599999999999999},"last":0.20399999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740631,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.20000000000000001,"5min":0.19800000000000001,"15min":0.20999999999999999},"min":{"1min":0.159,"5min":0.11799999999999999,"15min":0.11799999999999999},"max":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.436},"last":0.215},{"interface":"front","average":{"1min":0.22700000000000001,"5min":0.22500000000000001,"15min":0.23000000000000001},"min":{"1min":0.17599999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.33400000000000002,"5min":0.374,"15min":0.433},"last":0.23599999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.222,"15min":0.22900000000000001},"min":{"1min":0.184,"5min":0.16600000000000001,"15min":0.158},"max":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.47599999999999998},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.246,"5min":0.23899999999999999,"15min":0.246},"min":{"1min":0.20000000000000001,"5min":0.14399999999999999,"15min":0.14399999999999999},"max":{"1min":0.372,"5min":0.39600000000000002,"15min":0.45800000000000002},"last":0.249}]}]},{"osd":1,"up_from":8,"seq":34359740631,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.19600000000000001,"5min":0.20899999999999999,"15min":0.21099999999999999},"min":{"1min":0.152,"5min":0.152,"15min":0.13800000000000001},"max":{"1min":0.27100000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.20699999999999999},{"interface":"front","average":{"1min":0.22600000000000001,"5min":0.24099999999999999,"15min":0.24199999999999999},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.17299999999999999},"max":{"1min":0.30299999999999999,"5min":0.435,"15min":0.45300000000000001},"last":0.25700000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.222,"5min":0.23100000000000001,"15min":0.23200000000000001},"min":{"1min":0.185,"5min":0.17399999999999999,"15min":0.16300000000000001},"max":{"1min":0.28000000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"last":0.245},{"interface":"front","average":{"1min":0.24299999999999999,"5min":0.25800000000000001,"15min":0.25900000000000001},"min":{"1min":0.19600000000000001,"5min":0.184,"15min":0.18099999999999999},"max":{"1min":0.28000000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.28799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:07:26.005 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:26.005 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:26.006 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:26.006 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:26.007 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:26.007 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:26.007 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:26.008 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:26.008 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:26.008 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:26.009 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:26.009 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:07:26.557 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:26.555+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:26.861 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:26.860+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:27.574 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:27.572+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:27.872 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:27.870+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:28.614 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:28.612+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:28.890 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:28.888+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:29.611 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:29.609+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:29.871 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:29.869+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:30.609 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:30.607+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:30.877 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:30.875+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:31.645 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:31.643+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:31.892 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:31.891+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:32.606 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:32.605+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:32.923 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:32.921+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:33.630 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:33.628+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:33.901 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:33.899+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:34.631 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:34.628+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:34.878 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:34.876+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:35.663 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:35.661+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:35.851 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:35.849+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:36.680 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:36.679+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:36.859 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:36.857+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:37.672 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:37.670+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:37.851 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:37.849+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:38.668 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:38.666+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:38.877 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:38.875+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:39.639 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:39.637+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:39.924 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:39.922+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:40.643 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:40.641+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:40.912 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:40.910+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:41.637 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:41.635+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:41.903 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:41.901+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:42.599 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:42.597+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:42.871 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:42.868+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:43.561 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:43.558+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:43.877 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:43.874+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:44.513 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:44.510+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:44.917 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:44.914+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:45.502 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:45.499+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:45.887 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:45.884+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:46.010 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:07:46.300 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:07:46.302 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:07:46.312 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5806,"stamp":"2023-10-18T19:07:44.537517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001616"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":151,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451303+0000","last_change":"2023-10-18T15:59:17.900205+0000","last_active":"2023-10-18T17:02:28.451303+0000","last_peered":"2023-10-18T17:02:28.451303+0000","last_clean":"2023-10-18T17:02:28.451303+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T17:02:28.451303+0000","last_undegraded":"2023-10-18T17:02:28.451303+0000","last_fullsized":"2023-10-18T17:02:28.451303+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1993,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452064+0000","last_change":"2023-10-18T16:03:52.125315+0000","last_active":"2023-10-18T17:02:28.452064+0000","last_peered":"2023-10-18T17:02:28.452064+0000","last_clean":"2023-10-18T17:02:28.452064+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T17:02:28.452064+0000","last_undegraded":"2023-10-18T17:02:28.452064+0000","last_fullsized":"2023-10-18T17:02:28.452064+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740634,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.224,"15min":0.22900000000000001},"min":{"1min":0.14199999999999999,"5min":0.14199999999999999,"15min":0.14199999999999999},"max":{"1min":0.40999999999999998,"5min":0.45300000000000001,"15min":0.47999999999999998},"last":0.16400000000000001},{"interface":"front","average":{"1min":0.27300000000000002,"5min":0.26500000000000001,"15min":0.26400000000000001},"min":{"1min":0.187,"5min":0.16600000000000001,"15min":0.16},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.51000000000000001},"last":0.22800000000000001}]},{"osd":1,"last update":"Wed Oct 18 19:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.246,"5min":0.23699999999999999,"15min":0.23699999999999999},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.44,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.19800000000000001},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.24399999999999999,"15min":0.246},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.16400000000000001},"max":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.48599999999999999},"last":0.20499999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740635,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.191,"5min":0.19500000000000001,"15min":0.20899999999999999},"min":{"1min":0.161,"5min":0.11799999999999999,"15min":0.11799999999999999},"max":{"1min":0.25800000000000001,"5min":0.38300000000000001,"15min":0.42499999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.20799999999999999,"5min":0.221,"15min":0.22800000000000001},"min":{"1min":0.16900000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.253,"5min":0.374,"15min":0.433},"last":0.48399999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.218,"5min":0.222,"15min":0.22700000000000001},"min":{"1min":0.17599999999999999,"5min":0.16600000000000001,"15min":0.158},"max":{"1min":0.27100000000000002,"5min":0.41699999999999998,"15min":0.47599999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.24299999999999999,"5min":0.23899999999999999,"15min":0.245},"min":{"1min":0.20599999999999999,"5min":0.14399999999999999,"15min":0.14399999999999999},"max":{"1min":0.33000000000000002,"5min":0.39600000000000002,"15min":0.45800000000000002},"last":0.498}]}]},{"osd":1,"up_from":8,"seq":34359740635,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.19600000000000001,"5min":0.20899999999999999,"15min":0.21099999999999999},"min":{"1min":0.152,"5min":0.152,"15min":0.13800000000000001},"max":{"1min":0.27100000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.189},{"interface":"front","average":{"1min":0.22600000000000001,"5min":0.24099999999999999,"15min":0.24199999999999999},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.17299999999999999},"max":{"1min":0.30299999999999999,"5min":0.435,"15min":0.45300000000000001},"last":0.23400000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.222,"5min":0.23100000000000001,"15min":0.23200000000000001},"min":{"1min":0.185,"5min":0.17399999999999999,"15min":0.16300000000000001},"max":{"1min":0.28000000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"last":0.24299999999999999},{"interface":"front","average":{"1min":0.24299999999999999,"5min":0.25800000000000001,"15min":0.25900000000000001},"min":{"1min":0.19600000000000001,"5min":0.184,"15min":0.18099999999999999},"max":{"1min":0.28000000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.216}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:07:46.314 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:46.314 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:46.315 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:46.315 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:46.316 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:46.316 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:46.316 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:46.317 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:46.317 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:46.317 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:46.318 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:07:46.318 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:07:46.499 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:46.496+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:46.929 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:46.926+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:47.462 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:47.459+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:47.967 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:47.964+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:48.511 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:48.508+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:48.991 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:48.988+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:49.548 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:49.545+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:50.006 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:50.003+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:50.508 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:50.505+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:50.985 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:50.982+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:51.471 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:51.468+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:52.008 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:52.005+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:52.458 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:52.455+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:53.053 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:53.050+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:53.441 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:53.438+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:54.069 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:54.066+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:54.397 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:54.394+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:55.069 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:55.066+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:55.440 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:55.437+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:56.029 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:56.026+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:56.483 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:56.480+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:57.026 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:57.023+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:57.436 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:57.434+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:58.025 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:58.022+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:58.410 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:58.407+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:59.011 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:59.008+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:59.374 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:07:59.371+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:07:59.970 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:07:59.967+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:00.419 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:00.416+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:00.943 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:00.940+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:01.383 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:01.380+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:01.936 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:01.933+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:02.431 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:02.428+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:02.927 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:02.924+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:03.393 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:03.390+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:03.924 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:03.921+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:04.345 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:04.342+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:04.966 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:04.963+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:05.359 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:05.356+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:05.985 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:05.982+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:06.319 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:08:06.386 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:06.383+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:06.637 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:08:06.638 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:08:06.649 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5817,"stamp":"2023-10-18T19:08:06.540758+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001856"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":151,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451303+0000","last_change":"2023-10-18T15:59:17.900205+0000","last_active":"2023-10-18T17:02:28.451303+0000","last_peered":"2023-10-18T17:02:28.451303+0000","last_clean":"2023-10-18T17:02:28.451303+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T17:02:28.451303+0000","last_undegraded":"2023-10-18T17:02:28.451303+0000","last_fullsized":"2023-10-18T17:02:28.451303+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1993,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452064+0000","last_change":"2023-10-18T16:03:52.125315+0000","last_active":"2023-10-18T17:02:28.452064+0000","last_peered":"2023-10-18T17:02:28.452064+0000","last_clean":"2023-10-18T17:02:28.452064+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T17:02:28.452064+0000","last_undegraded":"2023-10-18T17:02:28.452064+0000","last_fullsized":"2023-10-18T17:02:28.452064+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740638,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.224,"15min":0.22900000000000001},"min":{"1min":0.14199999999999999,"5min":0.14199999999999999,"15min":0.14199999999999999},"max":{"1min":0.40999999999999998,"5min":0.45300000000000001,"15min":0.47999999999999998},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.27300000000000002,"5min":0.26500000000000001,"15min":0.26400000000000001},"min":{"1min":0.187,"5min":0.16600000000000001,"15min":0.16},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.51000000000000001},"last":0.26400000000000001}]},{"osd":1,"last update":"Wed Oct 18 19:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.246,"5min":0.23699999999999999,"15min":0.23699999999999999},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.44,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.25},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.24399999999999999,"15min":0.246},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.16400000000000001},"max":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.48599999999999999},"last":0.23699999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740640,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.191,"5min":0.19500000000000001,"15min":0.20899999999999999},"min":{"1min":0.161,"5min":0.11799999999999999,"15min":0.11799999999999999},"max":{"1min":0.25800000000000001,"5min":0.38300000000000001,"15min":0.42499999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.20799999999999999,"5min":0.221,"15min":0.22800000000000001},"min":{"1min":0.16900000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.253,"5min":0.374,"15min":0.433},"last":0.38300000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.218,"5min":0.222,"15min":0.22700000000000001},"min":{"1min":0.17599999999999999,"5min":0.16600000000000001,"15min":0.158},"max":{"1min":0.27100000000000002,"5min":0.41699999999999998,"15min":0.47599999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.24299999999999999,"5min":0.23899999999999999,"15min":0.245},"min":{"1min":0.20599999999999999,"5min":0.14399999999999999,"15min":0.14399999999999999},"max":{"1min":0.33000000000000002,"5min":0.39600000000000002,"15min":0.45800000000000002},"last":0.41199999999999998}]}]},{"osd":1,"up_from":8,"seq":34359740640,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.19600000000000001,"5min":0.20899999999999999,"15min":0.21099999999999999},"min":{"1min":0.152,"5min":0.152,"15min":0.13800000000000001},"max":{"1min":0.27100000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.20799999999999999},{"interface":"front","average":{"1min":0.22600000000000001,"5min":0.24099999999999999,"15min":0.24199999999999999},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.17299999999999999},"max":{"1min":0.30299999999999999,"5min":0.435,"15min":0.45300000000000001},"last":0.26100000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.222,"5min":0.23100000000000001,"15min":0.23200000000000001},"min":{"1min":0.185,"5min":0.17399999999999999,"15min":0.16300000000000001},"max":{"1min":0.28000000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"last":0.20000000000000001},{"interface":"front","average":{"1min":0.24299999999999999,"5min":0.25800000000000001,"15min":0.25900000000000001},"min":{"1min":0.19600000000000001,"5min":0.184,"15min":0.18099999999999999},"max":{"1min":0.28000000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.23899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:08:06.651 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:06.652 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:06.652 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:06.653 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:06.653 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:06.653 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:06.654 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:06.655 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:06.655 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:06.656 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:06.656 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:06.658 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:08:07.011 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:07.008+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:07.381 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:07.378+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:08.023 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:08.020+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:08.398 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:08.395+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:09.031 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:09.028+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:09.449 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:09.446+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:09.989 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:09.986+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:10.457 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:10.454+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:10.990 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:10.987+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:11.443 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:11.440+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:11.996 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:11.993+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:12.462 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:12.459+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:12.971 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:12.968+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:13.502 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:13.500+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:13.987 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:13.984+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:14.478 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:14.475+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:14.967 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:14.964+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:15.460 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:15.457+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:15.926 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:15.924+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:16.439 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:16.437+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:16.923 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:16.921+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:17.460 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:17.458+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:17.923 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:17.921+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:18.455 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:18.453+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:18.961 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:18.959+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:19.469 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:19.467+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:20.011 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:20.009+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:20.424 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:20.422+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:20.972 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:20.970+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:21.441 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:21.439+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:21.946 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:21.944+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:22.427 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:22.425+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:22.948 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:22.946+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:23.406 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:23.404+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:23.938 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:23.936+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:24.384 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:24.381+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:24.946 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:24.944+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:25.390 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:25.388+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:25.958 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:25.957+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:26.405 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:26.403+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:26.659 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:08:26.953 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:08:26.955 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:08:26.962 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:26.960+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:26.967 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5827,"stamp":"2023-10-18T19:08:26.543885+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001745"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":151,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451303+0000","last_change":"2023-10-18T15:59:17.900205+0000","last_active":"2023-10-18T17:02:28.451303+0000","last_peered":"2023-10-18T17:02:28.451303+0000","last_clean":"2023-10-18T17:02:28.451303+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T17:02:28.451303+0000","last_undegraded":"2023-10-18T17:02:28.451303+0000","last_fullsized":"2023-10-18T17:02:28.451303+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1993,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452064+0000","last_change":"2023-10-18T16:03:52.125315+0000","last_active":"2023-10-18T17:02:28.452064+0000","last_peered":"2023-10-18T17:02:28.452064+0000","last_clean":"2023-10-18T17:02:28.452064+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T17:02:28.452064+0000","last_undegraded":"2023-10-18T17:02:28.452064+0000","last_fullsized":"2023-10-18T17:02:28.452064+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740642,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.224,"15min":0.22900000000000001},"min":{"1min":0.14199999999999999,"5min":0.14199999999999999,"15min":0.14199999999999999},"max":{"1min":0.40999999999999998,"5min":0.45300000000000001,"15min":0.47999999999999998},"last":0.18099999999999999},{"interface":"front","average":{"1min":0.27300000000000002,"5min":0.26500000000000001,"15min":0.26400000000000001},"min":{"1min":0.187,"5min":0.16600000000000001,"15min":0.16},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.51000000000000001},"last":0.25}]},{"osd":1,"last update":"Wed Oct 18 19:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.246,"5min":0.23699999999999999,"15min":0.23699999999999999},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.44,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.24399999999999999,"15min":0.246},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.16400000000000001},"max":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.48599999999999999},"last":0.222}]}]},{"osd":0,"up_from":8,"seq":34359740644,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:08:25 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.20200000000000001,"15min":0.20999999999999999},"min":{"1min":0.157,"5min":0.13200000000000001,"15min":0.11799999999999999},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.186},{"interface":"front","average":{"1min":0.25,"5min":0.22600000000000001,"15min":0.22900000000000001},"min":{"1min":0.17299999999999999,"5min":0.16900000000000001,"15min":0.16600000000000001},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.219}]},{"osd":2,"last update":"Wed Oct 18 19:08:25 2023","interfaces":[{"interface":"back","average":{"1min":0.247,"5min":0.22800000000000001,"15min":0.22900000000000001},"min":{"1min":0.17899999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.47599999999999998},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.24399999999999999,"15min":0.246},"min":{"1min":0.20200000000000001,"5min":0.17199999999999999,"15min":0.14399999999999999},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.25700000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740644,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.23100000000000001,"5min":0.214,"15min":0.21299999999999999},"min":{"1min":0.17599999999999999,"5min":0.152,"15min":0.13800000000000001},"max":{"1min":0.38,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.20000000000000001},{"interface":"front","average":{"1min":0.26000000000000001,"5min":0.246,"15min":0.24299999999999999},"min":{"1min":0.214,"5min":0.17599999999999999,"15min":0.17299999999999999},"max":{"1min":0.441,"5min":0.441,"15min":0.45300000000000001},"last":0.23599999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"min":{"1min":0.18099999999999999,"5min":0.17399999999999999,"15min":0.16300000000000001},"max":{"1min":0.29399999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"last":0.21199999999999999},{"interface":"front","average":{"1min":0.27000000000000002,"5min":0.26200000000000001,"15min":0.26000000000000001},"min":{"1min":0.216,"5min":0.184,"15min":0.18099999999999999},"max":{"1min":0.41099999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.25800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:08:26.968 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:26.969 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:26.969 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:26.970 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:26.970 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:26.971 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:26.971 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:26.972 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:26.972 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:26.972 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:26.973 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:26.973 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:08:27.433 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:27.431+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:27.965 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:27.963+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:28.449 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:28.447+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:28.946 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:28.943+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:29.488 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:29.486+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:29.933 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:29.931+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:30.476 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:30.474+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:30.920 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:30.918+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:31.458 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:31.456+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:31.931 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:31.929+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:32.481 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:32.478+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:32.964 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:32.962+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:33.489 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:33.486+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:33.925 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:33.923+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:34.460 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:34.457+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:34.923 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:34.920+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:35.433 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:35.430+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:35.886 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:35.883+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:36.428 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:36.425+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:36.843 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:36.840+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:37.451 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:37.448+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:37.867 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:37.864+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:38.439 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:38.436+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:38.863 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:38.860+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:39.473 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:39.471+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:39.815 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:39.812+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:40.495 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:40.492+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:40.825 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:40.822+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:41.498 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:41.495+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:41.812 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:41.809+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:42.484 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:42.481+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:42.786 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:42.783+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:43.437 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:43.433+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:43.799 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:43.796+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:44.396 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:44.393+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:44.846 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:44.843+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:45.392 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:45.389+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:45.884 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:45.881+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:46.352 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:46.349+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:46.890 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:46.887+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:46.974 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:08:47.272 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:08:47.274 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:08:47.284 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5837,"stamp":"2023-10-18T19:08:46.546875+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001777"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":151,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451303+0000","last_change":"2023-10-18T15:59:17.900205+0000","last_active":"2023-10-18T17:02:28.451303+0000","last_peered":"2023-10-18T17:02:28.451303+0000","last_clean":"2023-10-18T17:02:28.451303+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T17:02:28.451303+0000","last_undegraded":"2023-10-18T17:02:28.451303+0000","last_fullsized":"2023-10-18T17:02:28.451303+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1993,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452064+0000","last_change":"2023-10-18T16:03:52.125315+0000","last_active":"2023-10-18T17:02:28.452064+0000","last_peered":"2023-10-18T17:02:28.452064+0000","last_clean":"2023-10-18T17:02:28.452064+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T17:02:28.452064+0000","last_undegraded":"2023-10-18T17:02:28.452064+0000","last_fullsized":"2023-10-18T17:02:28.452064+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740646,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:08:31 2023","interfaces":[{"interface":"back","average":{"1min":0.20999999999999999,"5min":0.221,"15min":0.22800000000000001},"min":{"1min":0.157,"5min":0.14199999999999999,"15min":0.14199999999999999},"max":{"1min":0.28399999999999997,"5min":0.45300000000000001,"15min":0.47999999999999998},"last":0.217},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.26500000000000001,"15min":0.26300000000000001},"min":{"1min":0.17999999999999999,"5min":0.16600000000000001,"15min":0.16},"max":{"1min":0.33200000000000002,"5min":0.47499999999999998,"15min":0.51000000000000001},"last":0.26800000000000002}]},{"osd":1,"last update":"Wed Oct 18 19:08:31 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.23999999999999999,"15min":0.23599999999999999},"min":{"1min":0.183,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.35099999999999998,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.255},{"interface":"front","average":{"1min":0.24099999999999999,"5min":0.247,"15min":0.245},"min":{"1min":0.192,"5min":0.17199999999999999,"15min":0.16900000000000001},"max":{"1min":0.32400000000000001,"5min":0.46100000000000002,"15min":0.48599999999999999},"last":0.245}]}]},{"osd":0,"up_from":8,"seq":34359740648,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:08:25 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.20200000000000001,"15min":0.20999999999999999},"min":{"1min":0.157,"5min":0.13200000000000001,"15min":0.11799999999999999},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.17699999999999999},{"interface":"front","average":{"1min":0.25,"5min":0.22600000000000001,"15min":0.22900000000000001},"min":{"1min":0.17299999999999999,"5min":0.16900000000000001,"15min":0.16600000000000001},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.222}]},{"osd":2,"last update":"Wed Oct 18 19:08:25 2023","interfaces":[{"interface":"back","average":{"1min":0.247,"5min":0.22800000000000001,"15min":0.22900000000000001},"min":{"1min":0.17899999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.47599999999999998},"last":0.216},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.24399999999999999,"15min":0.246},"min":{"1min":0.20200000000000001,"5min":0.17199999999999999,"15min":0.14399999999999999},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.28100000000000003}]}]},{"osd":1,"up_from":8,"seq":34359740648,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.23100000000000001,"5min":0.214,"15min":0.21299999999999999},"min":{"1min":0.17599999999999999,"5min":0.152,"15min":0.13800000000000001},"max":{"1min":0.38,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.16400000000000001},{"interface":"front","average":{"1min":0.26000000000000001,"5min":0.246,"15min":0.24299999999999999},"min":{"1min":0.214,"5min":0.17599999999999999,"15min":0.17299999999999999},"max":{"1min":0.441,"5min":0.441,"15min":0.45300000000000001},"last":0.189}]},{"osd":2,"last update":"Wed Oct 18 19:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"min":{"1min":0.18099999999999999,"5min":0.17399999999999999,"15min":0.16300000000000001},"max":{"1min":0.29399999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"last":0.17899999999999999},{"interface":"front","average":{"1min":0.27000000000000002,"5min":0.26200000000000001,"15min":0.26000000000000001},"min":{"1min":0.216,"5min":0.184,"15min":0.18099999999999999},"max":{"1min":0.41099999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.21299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:08:47.286 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:47.286 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:47.287 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:47.287 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:47.287 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:47.288 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:47.288 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:47.289 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:47.289 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:47.290 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:47.290 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:08:47.290 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.7 2023-10-18T19:08:47.312 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:47.309+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:47.623 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.7 on osd.1 to deep-scrub 2023-10-18T19:08:47.635 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-10-18T19:08:47.858 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:47.855+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:48.011 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-10-18T19:08:48.023 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.5 2023-10-18T19:08:48.343 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:48.340+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:48.344 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.5 on osd.1 to deep-scrub 2023-10-18T19:08:48.355 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.1 2023-10-18T19:08:48.692 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.1 on osd.0 to deep-scrub 2023-10-18T19:08:48.704 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.2 2023-10-18T19:08:48.848 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:48.845+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:49.001 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.2 on osd.1 to deep-scrub 2023-10-18T19:08:49.013 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 1.0 2023-10-18T19:08:49.306 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 1.0 on osd.1 to deep-scrub 2023-10-18T19:08:49.316 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-10-18T19:08:49.367 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:49.365+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:49.654 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-10-18T19:08:49.667 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.0 2023-10-18T19:08:49.858 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:49.855+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:49.968 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2023-10-18T19:08:49.979 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.3 2023-10-18T19:08:50.280 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.3 on osd.0 to deep-scrub 2023-10-18T19:08:50.292 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.2 2023-10-18T19:08:50.407 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:50.404+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:50.586 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.2 on osd.0 to deep-scrub 2023-10-18T19:08:50.597 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-10-18T19:08:50.838 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:50.835+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:50.892 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-10-18T19:08:50.903 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:08:51.377 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:51.374+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:51.816 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:51.813+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:52.421 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:52.418+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:52.824 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:52.821+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:53.383 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:53.380+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:53.817 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:53.814+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:54.387 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:54.384+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:54.782 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:54.779+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:55.402 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:55.399+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:55.826 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:55.823+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:56.360 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:56.357+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:56.784 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:56.781+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:57.320 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:57.317+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:57.766 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:57.763+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:58.318 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:58.315+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:58.788 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:58.785+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:59.273 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:08:59.270+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:08:59.747 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:08:59.744+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:00.254 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:00.251+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:00.758 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:00.755+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:01.296 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:01.293+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:01.725 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:01.722+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:02.275 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:02.272+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:02.705 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:02.702+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:03.278 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:03.275+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:03.688 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:03.685+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:04.292 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:04.289+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:04.670 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:04.667+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:05.247 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:05.244+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:05.683 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:05.680+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:06.242 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:06.240+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:06.694 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:06.692+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:07.233 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:07.231+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:07.711 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:07.709+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:08.280 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:08.277+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:08.677 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:08.675+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:09.274 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:09.272+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:09.636 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:09.634+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:10.308 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:10.306+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:10.588 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:10.586+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:10.905 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:09:11.203 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:09:11.205 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:09:11.217 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5849,"stamp":"2023-10-18T19:09:10.550547+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001741"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740651,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:08:31 2023","interfaces":[{"interface":"back","average":{"1min":0.20999999999999999,"5min":0.221,"15min":0.22800000000000001},"min":{"1min":0.157,"5min":0.14199999999999999,"15min":0.14199999999999999},"max":{"1min":0.28399999999999997,"5min":0.45300000000000001,"15min":0.47999999999999998},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.26500000000000001,"15min":0.26300000000000001},"min":{"1min":0.17999999999999999,"5min":0.16600000000000001,"15min":0.16},"max":{"1min":0.33200000000000002,"5min":0.47499999999999998,"15min":0.51000000000000001},"last":0.189}]},{"osd":1,"last update":"Wed Oct 18 19:08:31 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.23999999999999999,"15min":0.23599999999999999},"min":{"1min":0.183,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.35099999999999998,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.24299999999999999},{"interface":"front","average":{"1min":0.24099999999999999,"5min":0.247,"15min":0.245},"min":{"1min":0.192,"5min":0.17199999999999999,"15min":0.16900000000000001},"max":{"1min":0.32400000000000001,"5min":0.46100000000000002,"15min":0.48599999999999999},"last":0.20999999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740652,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:08:25 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.20200000000000001,"15min":0.20999999999999999},"min":{"1min":0.157,"5min":0.13200000000000001,"15min":0.11799999999999999},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.25,"5min":0.22600000000000001,"15min":0.22900000000000001},"min":{"1min":0.17299999999999999,"5min":0.16900000000000001,"15min":0.16600000000000001},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.23400000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:08:25 2023","interfaces":[{"interface":"back","average":{"1min":0.247,"5min":0.22800000000000001,"15min":0.22900000000000001},"min":{"1min":0.17899999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.47599999999999998},"last":0.214},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.24399999999999999,"15min":0.246},"min":{"1min":0.20200000000000001,"5min":0.17199999999999999,"15min":0.14399999999999999},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.24399999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740652,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.23100000000000001,"5min":0.214,"15min":0.21299999999999999},"min":{"1min":0.17599999999999999,"5min":0.152,"15min":0.13800000000000001},"max":{"1min":0.38,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.26000000000000001,"5min":0.246,"15min":0.24299999999999999},"min":{"1min":0.214,"5min":0.17599999999999999,"15min":0.17299999999999999},"max":{"1min":0.441,"5min":0.441,"15min":0.45300000000000001},"last":0.51000000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"min":{"1min":0.18099999999999999,"5min":0.17399999999999999,"15min":0.16300000000000001},"max":{"1min":0.29399999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.27000000000000002,"5min":0.26200000000000001,"15min":0.26000000000000001},"min":{"1min":0.216,"5min":0.184,"15min":0.18099999999999999},"max":{"1min":0.41099999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.48899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:09:11.219 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:11.220 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:11.220 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:11.221 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:11.221 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:11.221 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:11.222 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:11.222 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:11.223 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:11.223 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:11.223 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:09:11.294 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:11.292+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:11.616 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:11.614+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:12.254 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:12.252+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:12.627 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:12.625+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:13.260 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:13.258+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:13.623 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:13.621+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:14.256 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:14.254+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:14.575 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:14.573+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:15.296 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:15.294+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:15.626 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:15.624+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:16.249 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:16.247+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:16.581 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:16.579+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:17.284 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:17.282+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:17.608 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:17.606+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:18.247 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:18.244+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:18.607 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:18.605+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:19.264 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:19.262+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:19.629 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:19.628+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:20.250 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:20.248+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:20.661 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:20.659+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:21.288 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:21.286+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:21.706 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:21.704+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:22.283 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:22.281+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:22.740 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:22.738+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:23.321 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:23.319+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:23.730 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:23.727+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:24.272 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:24.269+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:24.708 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:24.705+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:25.295 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:25.292+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:25.749 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:25.746+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:26.332 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:26.329+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:26.759 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:26.756+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:27.364 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:27.361+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:27.733 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:27.730+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:28.321 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:28.318+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:28.710 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:28.707+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:29.316 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:29.313+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:29.705 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:29.702+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:30.348 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:30.344+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:30.682 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:30.679+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:31.225 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:09:31.376 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:31.373+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:31.517 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:09:31.519 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:09:31.530 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5859,"stamp":"2023-10-18T19:09:30.553581+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001825"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740655,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:08:31 2023","interfaces":[{"interface":"back","average":{"1min":0.20999999999999999,"5min":0.221,"15min":0.22800000000000001},"min":{"1min":0.157,"5min":0.14199999999999999,"15min":0.14199999999999999},"max":{"1min":0.28399999999999997,"5min":0.45300000000000001,"15min":0.47999999999999998},"last":0.17199999999999999},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.26500000000000001,"15min":0.26300000000000001},"min":{"1min":0.17999999999999999,"5min":0.16600000000000001,"15min":0.16},"max":{"1min":0.33200000000000002,"5min":0.47499999999999998,"15min":0.51000000000000001},"last":0.22500000000000001}]},{"osd":1,"last update":"Wed Oct 18 19:08:31 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.23999999999999999,"15min":0.23599999999999999},"min":{"1min":0.183,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.35099999999999998,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.24099999999999999,"5min":0.247,"15min":0.245},"min":{"1min":0.192,"5min":0.17199999999999999,"15min":0.16900000000000001},"max":{"1min":0.32400000000000001,"5min":0.46100000000000002,"15min":0.48599999999999999},"last":0.245}]}]},{"osd":0,"up_from":8,"seq":34359740656,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:08:25 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.20200000000000001,"15min":0.20999999999999999},"min":{"1min":0.157,"5min":0.13200000000000001,"15min":0.11799999999999999},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.25,"5min":0.22600000000000001,"15min":0.22900000000000001},"min":{"1min":0.17299999999999999,"5min":0.16900000000000001,"15min":0.16600000000000001},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.40600000000000003}]},{"osd":2,"last update":"Wed Oct 18 19:08:25 2023","interfaces":[{"interface":"back","average":{"1min":0.247,"5min":0.22800000000000001,"15min":0.22900000000000001},"min":{"1min":0.17899999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.47599999999999998},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.24399999999999999,"15min":0.246},"min":{"1min":0.20200000000000001,"5min":0.17199999999999999,"15min":0.14399999999999999},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.41399999999999998}]}]},{"osd":1,"up_from":8,"seq":34359740656,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:09:14 2023","interfaces":[{"interface":"back","average":{"1min":0.218,"5min":0.218,"15min":0.21199999999999999},"min":{"1min":0.155,"5min":0.152,"15min":0.13800000000000001},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.253,"5min":0.25,"15min":0.24299999999999999},"min":{"1min":0.189,"5min":0.17599999999999999,"15min":0.17299999999999999},"max":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.25700000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:09:14 2023","interfaces":[{"interface":"back","average":{"1min":0.246,"5min":0.23799999999999999,"15min":0.23400000000000001},"min":{"1min":0.17899999999999999,"5min":0.17399999999999999,"15min":0.16900000000000001},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.26600000000000001,"5min":0.26700000000000002,"15min":0.25900000000000001},"min":{"1min":0.19800000000000001,"5min":0.19600000000000001,"15min":0.18099999999999999},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.29199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:09:31.531 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:31.532 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:31.532 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:31.533 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:31.533 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:31.533 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:31.534 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:31.534 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:31.535 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:31.535 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:31.535 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:09:31.647 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:31.644+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:32.347 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:32.344+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:32.646 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:32.643+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:33.386 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:33.383+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:33.623 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:33.620+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:34.359 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:34.356+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:34.625 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:34.622+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:35.353 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:35.350+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:35.598 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:35.595+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:36.354 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:36.351+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:36.574 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:36.571+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:37.337 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:37.334+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:37.558 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:37.555+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:38.379 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:38.376+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:38.562 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:38.559+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:39.378 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:39.375+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:39.573 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:39.570+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:40.373 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:40.370+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:40.545 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:40.542+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:41.377 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:41.374+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:41.503 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:41.500+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:42.350 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:42.347+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:42.459 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:42.457+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:43.395 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:43.392+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:43.477 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:43.474+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:44.378 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:44.375+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:44.479 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:44.477+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:45.386 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:45.383+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:45.474 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:45.471+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:46.349 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:46.346+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:46.431 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:46.428+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:47.343 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:47.340+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:47.469 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:47.466+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:48.324 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:48.321+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:48.489 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:48.486+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:49.341 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:49.338+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:49.476 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:49.473+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:50.342 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:50.339+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:50.478 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:50.475+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:51.390 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:51.387+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:51.496 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:51.493+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:51.536 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:09:51.830 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:09:51.831 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:09:51.842 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5869,"stamp":"2023-10-18T19:09:50.556374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001759"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740659,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:09:31 2023","interfaces":[{"interface":"back","average":{"1min":0.248,"5min":0.22700000000000001,"15min":0.23000000000000001},"min":{"1min":0.16400000000000001,"5min":0.14199999999999999,"15min":0.14199999999999999},"max":{"1min":0.40200000000000002,"5min":0.45300000000000001,"15min":0.47999999999999998},"last":0.215},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.26600000000000001,"15min":0.26300000000000001},"min":{"1min":0.189,"5min":0.17999999999999999,"15min":0.16},"max":{"1min":0.46400000000000002,"5min":0.47499999999999998,"15min":0.51000000000000001},"last":0.20599999999999999}]},{"osd":1,"last update":"Wed Oct 18 19:09:31 2023","interfaces":[{"interface":"back","average":{"1min":0.23599999999999999,"5min":0.23899999999999999,"15min":0.23599999999999999},"min":{"1min":0.193,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.27900000000000003,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.183},{"interface":"front","average":{"1min":0.24299999999999999,"5min":0.248,"15min":0.245},"min":{"1min":0.20999999999999999,"5min":0.17199999999999999,"15min":0.16900000000000001},"max":{"1min":0.29799999999999999,"5min":0.46100000000000002,"15min":0.48199999999999998},"last":0.23000000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740660,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:09:28 2023","interfaces":[{"interface":"back","average":{"1min":0.217,"5min":0.20699999999999999,"15min":0.20999999999999999},"min":{"1min":0.153,"5min":0.13200000000000001,"15min":0.11799999999999999},"max":{"1min":0.42699999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.214},{"interface":"front","average":{"1min":0.23499999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"min":{"1min":0.17699999999999999,"5min":0.16900000000000001,"15min":0.16600000000000001},"max":{"1min":0.40600000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.27900000000000003}]},{"osd":2,"last update":"Wed Oct 18 19:09:28 2023","interfaces":[{"interface":"back","average":{"1min":0.23499999999999999,"5min":0.23200000000000001,"15min":0.22900000000000001},"min":{"1min":0.18099999999999999,"5min":0.17000000000000001,"15min":0.16600000000000001},"max":{"1min":0.434,"5min":0.44600000000000001,"15min":0.47599999999999998},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.253,"5min":0.249,"15min":0.246},"min":{"1min":0.19,"5min":0.17199999999999999,"15min":0.14399999999999999},"max":{"1min":0.42599999999999999,"5min":0.498,"15min":0.498},"last":0.26400000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740660,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:09:14 2023","interfaces":[{"interface":"back","average":{"1min":0.218,"5min":0.218,"15min":0.21199999999999999},"min":{"1min":0.155,"5min":0.152,"15min":0.13800000000000001},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.188},{"interface":"front","average":{"1min":0.253,"5min":0.25,"15min":0.24299999999999999},"min":{"1min":0.189,"5min":0.17599999999999999,"15min":0.17299999999999999},"max":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.215}]},{"osd":2,"last update":"Wed Oct 18 19:09:14 2023","interfaces":[{"interface":"back","average":{"1min":0.246,"5min":0.23799999999999999,"15min":0.23400000000000001},"min":{"1min":0.17899999999999999,"5min":0.17399999999999999,"15min":0.16900000000000001},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.26600000000000001,"5min":0.26700000000000002,"15min":0.25900000000000001},"min":{"1min":0.19800000000000001,"5min":0.19600000000000001,"15min":0.18099999999999999},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.24199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:09:51.844 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:51.844 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:51.845 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:51.845 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:51.846 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:51.846 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:51.846 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:51.847 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:51.847 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:51.848 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:09:51.848 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:09:52.396 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:52.393+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:52.539 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:52.536+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:53.425 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:53.422+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:53.506 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:53.503+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:54.425 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:54.421+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:54.529 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:54.526+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:55.419 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:55.416+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:55.578 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:55.575+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:56.385 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:56.383+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:56.555 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:56.552+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:57.389 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:57.387+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:57.600 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:57.598+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:58.393 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:58.391+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:58.649 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:58.647+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:59.435 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:09:59.433+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:09:59.676 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:09:59.674+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:00.433 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:00.431+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:00.701 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:00.699+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:01.475 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:01.473+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:01.741 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:01.739+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:02.439 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:02.437+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:02.743 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:02.741+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:03.393 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:03.390+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:03.786 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:03.784+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:04.403 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:04.401+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:04.776 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:04.774+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:05.356 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:05.354+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:05.815 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:05.813+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:06.390 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:06.389+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:06.773 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:06.771+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:07.343 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:07.341+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:07.771 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:07.769+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:08.324 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:08.322+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:08.762 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:08.760+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:09.316 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:09.314+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:09.748 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:09.746+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:10.277 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:10.275+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:10.766 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:10.764+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:11.290 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:11.288+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:11.746 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:11.744+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:11.849 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:10:12.148 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:10:12.149 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:10:12.161 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5879,"stamp":"2023-10-18T19:10:10.559308+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001851"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740663,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:09:31 2023","interfaces":[{"interface":"back","average":{"1min":0.248,"5min":0.22700000000000001,"15min":0.23000000000000001},"min":{"1min":0.16400000000000001,"5min":0.14199999999999999,"15min":0.14199999999999999},"max":{"1min":0.40200000000000002,"5min":0.45300000000000001,"15min":0.47999999999999998},"last":0.17799999999999999},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.26600000000000001,"15min":0.26300000000000001},"min":{"1min":0.189,"5min":0.17999999999999999,"15min":0.16},"max":{"1min":0.46400000000000002,"5min":0.47499999999999998,"15min":0.51000000000000001},"last":0.25600000000000001}]},{"osd":1,"last update":"Wed Oct 18 19:09:31 2023","interfaces":[{"interface":"back","average":{"1min":0.23599999999999999,"5min":0.23899999999999999,"15min":0.23599999999999999},"min":{"1min":0.193,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.27900000000000003,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.20699999999999999},{"interface":"front","average":{"1min":0.24299999999999999,"5min":0.248,"15min":0.245},"min":{"1min":0.20999999999999999,"5min":0.17199999999999999,"15min":0.16900000000000001},"max":{"1min":0.29799999999999999,"5min":0.46100000000000002,"15min":0.48199999999999998},"last":0.24399999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740664,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:09:28 2023","interfaces":[{"interface":"back","average":{"1min":0.217,"5min":0.20699999999999999,"15min":0.20999999999999999},"min":{"1min":0.153,"5min":0.13200000000000001,"15min":0.11799999999999999},"max":{"1min":0.42699999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.17299999999999999},{"interface":"front","average":{"1min":0.23499999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"min":{"1min":0.17699999999999999,"5min":0.16900000000000001,"15min":0.16600000000000001},"max":{"1min":0.40600000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.20599999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:09:28 2023","interfaces":[{"interface":"back","average":{"1min":0.23499999999999999,"5min":0.23200000000000001,"15min":0.22900000000000001},"min":{"1min":0.18099999999999999,"5min":0.17000000000000001,"15min":0.16600000000000001},"max":{"1min":0.434,"5min":0.44600000000000001,"15min":0.47599999999999998},"last":0.217},{"interface":"front","average":{"1min":0.253,"5min":0.249,"15min":0.246},"min":{"1min":0.19,"5min":0.17199999999999999,"15min":0.14399999999999999},"max":{"1min":0.42599999999999999,"5min":0.498,"15min":0.498},"last":0.26100000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740664,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:09:14 2023","interfaces":[{"interface":"back","average":{"1min":0.218,"5min":0.218,"15min":0.21199999999999999},"min":{"1min":0.155,"5min":0.152,"15min":0.13800000000000001},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.22},{"interface":"front","average":{"1min":0.253,"5min":0.25,"15min":0.24299999999999999},"min":{"1min":0.189,"5min":0.17599999999999999,"15min":0.17299999999999999},"max":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.26400000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:09:14 2023","interfaces":[{"interface":"back","average":{"1min":0.246,"5min":0.23799999999999999,"15min":0.23400000000000001},"min":{"1min":0.17899999999999999,"5min":0.17399999999999999,"15min":0.16900000000000001},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.26600000000000001,"5min":0.26700000000000002,"15min":0.25900000000000001},"min":{"1min":0.19800000000000001,"5min":0.19600000000000001,"15min":0.18099999999999999},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.29599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:10:12.163 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:12.163 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:12.163 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:12.164 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:12.164 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:12.165 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:12.165 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:12.165 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:12.166 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:12.166 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:12.167 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:10:12.327 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:12.325+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:12.715 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:12.713+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:13.287 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:13.285+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:13.667 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:13.665+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:14.239 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:14.237+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:14.712 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:14.709+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:15.253 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:15.250+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:15.713 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:15.710+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:16.238 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:16.235+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:16.761 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:16.758+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:17.209 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:17.206+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:17.729 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:17.726+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:18.176 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:18.173+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:18.756 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:18.754+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:19.151 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:19.148+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:19.724 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:19.721+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:20.201 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:20.198+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:20.684 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:20.681+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:21.221 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:21.218+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:21.666 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:21.664+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:22.256 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:22.253+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:22.667 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:22.664+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:23.289 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:23.286+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:23.640 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:23.637+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:24.259 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:24.256+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:24.621 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:24.618+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:25.224 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:25.222+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:25.605 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:25.603+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:26.217 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:26.214+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:26.574 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:26.571+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:27.266 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:27.263+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:27.616 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:27.613+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:28.229 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:28.226+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:28.625 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:28.622+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:29.227 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:29.224+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:29.613 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:29.610+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:30.256 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:30.253+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:30.575 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:30.572+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:31.275 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:31.271+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:31.564 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:31.561+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:32.167 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:10:32.298 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:32.295+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:32.460 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:10:32.462 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:10:32.473 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5889,"stamp":"2023-10-18T19:10:30.562627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001850"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740667,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:09:31 2023","interfaces":[{"interface":"back","average":{"1min":0.248,"5min":0.22700000000000001,"15min":0.23000000000000001},"min":{"1min":0.16400000000000001,"5min":0.14199999999999999,"15min":0.14199999999999999},"max":{"1min":0.40200000000000002,"5min":0.45300000000000001,"15min":0.47999999999999998},"last":0.19500000000000001},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.26600000000000001,"15min":0.26300000000000001},"min":{"1min":0.189,"5min":0.17999999999999999,"15min":0.16},"max":{"1min":0.46400000000000002,"5min":0.47499999999999998,"15min":0.51000000000000001},"last":0.25700000000000001}]},{"osd":1,"last update":"Wed Oct 18 19:09:31 2023","interfaces":[{"interface":"back","average":{"1min":0.23599999999999999,"5min":0.23899999999999999,"15min":0.23599999999999999},"min":{"1min":0.193,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.27900000000000003,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.24299999999999999,"5min":0.248,"15min":0.245},"min":{"1min":0.20999999999999999,"5min":0.17199999999999999,"15min":0.16900000000000001},"max":{"1min":0.29799999999999999,"5min":0.46100000000000002,"15min":0.48199999999999998},"last":0.24299999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740668,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:09:28 2023","interfaces":[{"interface":"back","average":{"1min":0.217,"5min":0.20699999999999999,"15min":0.20999999999999999},"min":{"1min":0.153,"5min":0.13200000000000001,"15min":0.11799999999999999},"max":{"1min":0.42699999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.17100000000000001},{"interface":"front","average":{"1min":0.23499999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"min":{"1min":0.17699999999999999,"5min":0.16900000000000001,"15min":0.16600000000000001},"max":{"1min":0.40600000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.215}]},{"osd":2,"last update":"Wed Oct 18 19:09:28 2023","interfaces":[{"interface":"back","average":{"1min":0.23499999999999999,"5min":0.23200000000000001,"15min":0.22900000000000001},"min":{"1min":0.18099999999999999,"5min":0.17000000000000001,"15min":0.16600000000000001},"max":{"1min":0.434,"5min":0.44600000000000001,"15min":0.47599999999999998},"last":0.20699999999999999},{"interface":"front","average":{"1min":0.253,"5min":0.249,"15min":0.246},"min":{"1min":0.19,"5min":0.17199999999999999,"15min":0.14399999999999999},"max":{"1min":0.42599999999999999,"5min":0.498,"15min":0.498},"last":0.23000000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740668,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:10:17 2023","interfaces":[{"interface":"back","average":{"1min":0.224,"5min":0.22,"15min":0.21199999999999999},"min":{"1min":0.16700000000000001,"5min":0.152,"15min":0.13800000000000001},"max":{"1min":0.41899999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.254,"5min":0.253,"15min":0.24399999999999999},"min":{"1min":0.19900000000000001,"5min":0.17599999999999999,"15min":0.17299999999999999},"max":{"1min":0.44800000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.22500000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:10:17 2023","interfaces":[{"interface":"back","average":{"1min":0.252,"5min":0.24199999999999999,"15min":0.23499999999999999},"min":{"1min":0.186,"5min":0.17899999999999999,"15min":0.16900000000000001},"max":{"1min":0.41099999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.247},{"interface":"front","average":{"1min":0.27700000000000002,"5min":0.26800000000000002,"15min":0.26100000000000001},"min":{"1min":0.222,"5min":0.19600000000000001,"15min":0.18099999999999999},"max":{"1min":0.47099999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.28799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:10:32.474 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:32.475 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:32.475 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:32.476 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:32.476 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:32.476 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:32.477 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:32.477 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:32.478 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:32.478 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:32.478 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:10:32.581 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:32.578+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:33.335 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:33.332+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:33.583 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:33.580+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:34.349 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:34.346+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:34.631 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:34.628+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:35.352 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:35.349+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:35.610 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:35.607+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:36.327 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:36.324+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:36.573 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:36.570+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:37.291 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:37.288+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:37.551 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:37.548+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:38.257 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:38.254+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:38.601 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:38.598+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:39.241 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:39.238+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:39.626 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:39.623+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:40.286 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:40.283+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:40.634 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:40.631+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:41.328 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:41.325+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:41.669 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:41.666+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:42.318 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:42.315+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:42.712 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:42.709+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:43.289 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:43.286+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:43.675 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:43.672+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:44.324 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:44.321+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:44.702 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:44.700+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:45.324 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:45.321+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:45.689 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:45.686+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:46.287 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:46.284+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:46.641 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:46.638+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:47.322 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:47.319+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:47.657 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:47.655+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:48.273 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:48.271+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:48.615 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:48.613+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:49.315 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:49.313+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:49.613 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:49.611+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:50.333 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:50.331+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:50.660 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:50.658+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:51.344 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:51.342+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:51.634 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:51.632+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:52.335 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:52.333+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:52.480 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:10:52.657 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:52.655+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:52.822 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:10:52.824 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:10:52.835 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5900,"stamp":"2023-10-18T19:10:52.565652+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001636"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740672,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.22900000000000001,"15min":0.22700000000000001},"min":{"1min":0.17699999999999999,"5min":0.14199999999999999,"15min":0.14199999999999999},"max":{"1min":0.443,"5min":0.45300000000000001,"15min":0.47999999999999998},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.26800000000000002,"5min":0.26700000000000002,"15min":0.26200000000000001},"min":{"1min":0.189,"5min":0.17999999999999999,"15min":0.16},"max":{"1min":0.46400000000000002,"5min":0.47499999999999998,"15min":0.51000000000000001},"last":0.24299999999999999}]},{"osd":1,"last update":"Wed Oct 18 19:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.24399999999999999,"5min":0.24099999999999999,"15min":0.23499999999999999},"min":{"1min":0.183,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.41199999999999998,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.223},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.25,"15min":0.245},"min":{"1min":0.20899999999999999,"5min":0.17199999999999999,"15min":0.16900000000000001},"max":{"1min":0.45100000000000001,"5min":0.46100000000000002,"15min":0.48199999999999998},"last":0.23100000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740673,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.20499999999999999,"5min":0.20899999999999999,"15min":0.20699999999999999},"min":{"1min":0.154,"5min":0.153,"15min":0.11799999999999999},"max":{"1min":0.307,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.24199999999999999,"5min":0.23200000000000001,"15min":0.22800000000000001},"min":{"1min":0.183,"5min":0.16900000000000001,"15min":0.16600000000000001},"max":{"1min":0.34000000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.318}]},{"osd":2,"last update":"Wed Oct 18 19:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.23200000000000001,"15min":0.22700000000000001},"min":{"1min":0.19500000000000001,"5min":0.17599999999999999,"15min":0.16600000000000001},"max":{"1min":0.35499999999999998,"5min":0.44600000000000001,"15min":0.44700000000000001},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.252,"5min":0.251,"15min":0.245},"min":{"1min":0.188,"5min":0.188,"15min":0.14399999999999999},"max":{"1min":0.41799999999999998,"5min":0.498,"15min":0.498},"last":0.34499999999999997}]}]},{"osd":1,"up_from":8,"seq":34359740673,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:10:17 2023","interfaces":[{"interface":"back","average":{"1min":0.224,"5min":0.22,"15min":0.21199999999999999},"min":{"1min":0.16700000000000001,"5min":0.152,"15min":0.13800000000000001},"max":{"1min":0.41899999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.19400000000000001},{"interface":"front","average":{"1min":0.254,"5min":0.253,"15min":0.24399999999999999},"min":{"1min":0.19900000000000001,"5min":0.17599999999999999,"15min":0.17299999999999999},"max":{"1min":0.44800000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.22800000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:10:17 2023","interfaces":[{"interface":"back","average":{"1min":0.252,"5min":0.24199999999999999,"15min":0.23499999999999999},"min":{"1min":0.186,"5min":0.17899999999999999,"15min":0.16900000000000001},"max":{"1min":0.41099999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.21199999999999999},{"interface":"front","average":{"1min":0.27700000000000002,"5min":0.26800000000000002,"15min":0.26100000000000001},"min":{"1min":0.222,"5min":0.19600000000000001,"15min":0.18099999999999999},"max":{"1min":0.47099999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.25900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:10:52.837 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:52.837 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:52.838 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:52.838 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:52.838 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:52.839 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:52.839 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:52.839 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:52.840 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:52.840 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:10:52.841 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:10:53.350 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:53.349+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:53.678 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:53.676+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:54.302 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:54.301+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:54.697 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:54.695+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:55.338 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:55.337+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:55.725 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:55.723+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:56.355 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:56.353+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:56.773 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:56.771+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:57.326 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:57.324+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:57.774 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:57.773+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:58.367 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:58.365+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:58.749 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:58.747+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:59.415 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:10:59.413+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:10:59.784 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:10:59.782+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:00.389 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:00.387+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:00.825 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:00.823+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:01.416 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:01.414+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:01.855 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:01.853+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:02.426 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:02.424+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:02.830 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:02.829+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:03.456 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:03.454+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:03.859 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:03.857+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:04.459 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:04.457+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:04.871 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:04.869+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:05.461 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:05.458+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:05.860 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:05.857+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:06.496 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:06.493+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:06.898 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:06.896+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:07.457 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:07.454+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:07.873 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:07.870+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:08.471 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:08.467+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:08.897 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:08.894+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:09.486 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:09.483+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:09.878 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:09.875+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:10.499 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:10.496+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:10.873 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:10.870+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:11.502 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:11.499+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:11.877 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:11.874+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:12.509 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:12.506+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:12.841 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:11:12.915 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:12.912+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:13.175 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:11:13.177 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:11:13.189 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5910,"stamp":"2023-10-18T19:11:12.568725+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001821"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740676,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.22900000000000001,"15min":0.22700000000000001},"min":{"1min":0.17699999999999999,"5min":0.14199999999999999,"15min":0.14199999999999999},"max":{"1min":0.443,"5min":0.45300000000000001,"15min":0.47999999999999998},"last":0.20699999999999999},{"interface":"front","average":{"1min":0.26800000000000002,"5min":0.26700000000000002,"15min":0.26200000000000001},"min":{"1min":0.189,"5min":0.17999999999999999,"15min":0.16},"max":{"1min":0.46400000000000002,"5min":0.47499999999999998,"15min":0.51000000000000001},"last":0.26800000000000002}]},{"osd":1,"last update":"Wed Oct 18 19:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.24399999999999999,"5min":0.24099999999999999,"15min":0.23499999999999999},"min":{"1min":0.183,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.41199999999999998,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.24299999999999999},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.25,"15min":0.245},"min":{"1min":0.20899999999999999,"5min":0.17199999999999999,"15min":0.16900000000000001},"max":{"1min":0.45100000000000001,"5min":0.46100000000000002,"15min":0.48199999999999998},"last":0.28299999999999997}]}]},{"osd":0,"up_from":8,"seq":34359740677,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.20499999999999999,"5min":0.20899999999999999,"15min":0.20699999999999999},"min":{"1min":0.154,"5min":0.153,"15min":0.11799999999999999},"max":{"1min":0.307,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.24199999999999999,"5min":0.23200000000000001,"15min":0.22800000000000001},"min":{"1min":0.183,"5min":0.16900000000000001,"15min":0.16600000000000001},"max":{"1min":0.34000000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.215}]},{"osd":2,"last update":"Wed Oct 18 19:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.23200000000000001,"15min":0.22700000000000001},"min":{"1min":0.19500000000000001,"5min":0.17599999999999999,"15min":0.16600000000000001},"max":{"1min":0.35499999999999998,"5min":0.44600000000000001,"15min":0.44700000000000001},"last":0.254},{"interface":"front","average":{"1min":0.252,"5min":0.251,"15min":0.245},"min":{"1min":0.188,"5min":0.188,"15min":0.14399999999999999},"max":{"1min":0.41799999999999998,"5min":0.498,"15min":0.498},"last":0.20599999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740677,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:10:17 2023","interfaces":[{"interface":"back","average":{"1min":0.224,"5min":0.22,"15min":0.21199999999999999},"min":{"1min":0.16700000000000001,"5min":0.152,"15min":0.13800000000000001},"max":{"1min":0.41899999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.17899999999999999},{"interface":"front","average":{"1min":0.254,"5min":0.253,"15min":0.24399999999999999},"min":{"1min":0.19900000000000001,"5min":0.17599999999999999,"15min":0.17299999999999999},"max":{"1min":0.44800000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.20999999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:10:17 2023","interfaces":[{"interface":"back","average":{"1min":0.252,"5min":0.24199999999999999,"15min":0.23499999999999999},"min":{"1min":0.186,"5min":0.17899999999999999,"15min":0.16900000000000001},"max":{"1min":0.41099999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.20100000000000001},{"interface":"front","average":{"1min":0.27700000000000002,"5min":0.26800000000000002,"15min":0.26100000000000001},"min":{"1min":0.222,"5min":0.19600000000000001,"15min":0.18099999999999999},"max":{"1min":0.47099999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.23400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:11:13.190 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:13.191 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:13.191 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:13.191 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:13.192 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:13.192 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:13.193 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:13.193 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:13.193 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:13.194 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:13.194 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.7 2023-10-18T19:11:13.531 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.7 on osd.1 to deep-scrub 2023-10-18T19:11:13.542 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-10-18T19:11:13.545 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:13.542+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:13.880 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-10-18T19:11:13.891 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.5 2023-10-18T19:11:13.901 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:13.898+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:14.189 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.5 on osd.1 to deep-scrub 2023-10-18T19:11:14.201 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.1 2023-10-18T19:11:14.537 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:14.534+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:14.567 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.1 on osd.0 to deep-scrub 2023-10-18T19:11:14.578 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.2 2023-10-18T19:11:14.871 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.2 on osd.1 to deep-scrub 2023-10-18T19:11:14.883 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-10-18T19:11:14.898 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:14.895+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:15.179 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-10-18T19:11:15.190 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.0 2023-10-18T19:11:15.527 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:15.524+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:15.532 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2023-10-18T19:11:15.545 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.3 2023-10-18T19:11:15.843 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.3 on osd.0 to deep-scrub 2023-10-18T19:11:15.855 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.2 2023-10-18T19:11:15.867 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:15.864+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:16.154 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.2 on osd.0 to deep-scrub 2023-10-18T19:11:16.165 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-10-18T19:11:16.506 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-10-18T19:11:16.517 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:11:16.576 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:16.573+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:16.835 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:16.832+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:17.593 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:17.590+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:17.877 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:17.874+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:18.617 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:18.614+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:18.865 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:18.862+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:19.654 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:19.651+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:19.820 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:19.817+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:20.630 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:20.627+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:20.825 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:20.822+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:21.671 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:21.668+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:21.809 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:21.806+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:22.651 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:22.648+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:22.835 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:22.832+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:23.668 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:23.666+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:23.851 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:23.848+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:24.691 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:24.688+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:24.844 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:24.841+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:25.661 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:25.658+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:25.823 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:25.820+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:26.614 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:26.611+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:26.834 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:26.831+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:27.659 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:27.656+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:27.829 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:27.826+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:28.649 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:28.646+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:28.856 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:28.853+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:29.632 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:29.629+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:29.869 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:29.866+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:30.677 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:30.674+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:30.890 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:30.887+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:31.658 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:31.655+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:31.906 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:31.903+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:32.695 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:32.692+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:32.933 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:32.930+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:33.659 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:33.656+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:33.894 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:33.891+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:34.615 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:34.612+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:34.902 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:34.898+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:35.601 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:35.598+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:35.936 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:35.934+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:36.519 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:11:36.588 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:36.585+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:36.819 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:11:36.821 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:11:36.832 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5922,"stamp":"2023-10-18T19:11:36.572490+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001795"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740680,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.22900000000000001,"15min":0.22700000000000001},"min":{"1min":0.17699999999999999,"5min":0.14199999999999999,"15min":0.14199999999999999},"max":{"1min":0.443,"5min":0.45300000000000001,"15min":0.47999999999999998},"last":0.17599999999999999},{"interface":"front","average":{"1min":0.26800000000000002,"5min":0.26700000000000002,"15min":0.26200000000000001},"min":{"1min":0.189,"5min":0.17999999999999999,"15min":0.16},"max":{"1min":0.46400000000000002,"5min":0.47499999999999998,"15min":0.51000000000000001},"last":0.23599999999999999}]},{"osd":1,"last update":"Wed Oct 18 19:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.24399999999999999,"5min":0.24099999999999999,"15min":0.23499999999999999},"min":{"1min":0.183,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.41199999999999998,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.22},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.25,"15min":0.245},"min":{"1min":0.20899999999999999,"5min":0.17199999999999999,"15min":0.16900000000000001},"max":{"1min":0.45100000000000001,"5min":0.46100000000000002,"15min":0.48199999999999998},"last":0.20999999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740682,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:11:34 2023","interfaces":[{"interface":"back","average":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.20699999999999999},"min":{"1min":0.156,"5min":0.153,"15min":0.11799999999999999},"max":{"1min":0.36799999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.224,"5min":0.23100000000000001,"15min":0.22800000000000001},"min":{"1min":0.156,"5min":0.156,"15min":0.156},"max":{"1min":0.38300000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.21099999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:11:34 2023","interfaces":[{"interface":"back","average":{"1min":0.23000000000000001,"5min":0.23300000000000001,"15min":0.22800000000000001},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.39500000000000002,"5min":0.44600000000000001,"15min":0.44700000000000001},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.23699999999999999,"5min":0.249,"15min":0.24399999999999999},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.14399999999999999},"max":{"1min":0.40300000000000002,"5min":0.498,"15min":0.498},"last":0.221}]}]},{"osd":1,"up_from":8,"seq":34359740682,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:11:18 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.22,"15min":0.214},"min":{"1min":0.16700000000000001,"5min":0.152,"15min":0.13800000000000001},"max":{"1min":0.42199999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.19},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.249,"15min":0.245},"min":{"1min":0.19900000000000001,"5min":0.17599999999999999,"15min":0.17299999999999999},"max":{"1min":0.47399999999999998,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.22}]},{"osd":2,"last update":"Wed Oct 18 19:11:18 2023","interfaces":[{"interface":"back","average":{"1min":0.24399999999999999,"5min":0.23899999999999999,"15min":0.23599999999999999},"min":{"1min":0.20100000000000001,"5min":0.17899999999999999,"15min":0.16900000000000001},"max":{"1min":0.42999999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.21199999999999999},{"interface":"front","average":{"1min":0.27900000000000003,"5min":0.26700000000000002,"15min":0.26300000000000001},"min":{"1min":0.22800000000000001,"5min":0.19600000000000001,"15min":0.18099999999999999},"max":{"1min":0.45100000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.24299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:11:36.834 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:36.834 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:36.835 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:36.835 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:36.836 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:36.836 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:36.836 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:36.837 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:36.837 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:36.838 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:36.838 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:11:36.896 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:36.893+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:37.588 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:37.585+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:37.861 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:37.858+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:38.540 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:38.538+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:38.895 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:38.893+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:39.573 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:39.571+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:39.929 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:39.927+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:40.600 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:40.598+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:40.897 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:40.895+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:41.646 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:41.644+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:41.924 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:41.922+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:42.659 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:42.657+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:42.920 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:42.918+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:43.638 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:43.636+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:43.898 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:43.896+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:44.594 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:44.591+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:44.912 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:44.909+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:45.564 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:45.562+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:45.902 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:45.900+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:46.584 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:46.582+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:46.943 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:46.941+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:47.538 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:47.536+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:47.937 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:47.935+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:48.543 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:48.541+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:48.888 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:48.886+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:49.580 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:49.578+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:49.912 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:49.910+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:50.594 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:50.592+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:50.943 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:50.941+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:51.613 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:51.611+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:51.966 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:51.964+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:52.636 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:52.634+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:52.966 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:52.964+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:53.630 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:53.628+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:53.987 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:53.986+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:54.666 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:54.664+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:55.024 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:55.022+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:55.710 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:55.707+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:56.056 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:56.053+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:56.686 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:56.683+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:56.839 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:11:57.019 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:57.016+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:57.131 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:11:57.133 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:11:57.144 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5932,"stamp":"2023-10-18T19:11:56.575994+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001873"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740684,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.20499999999999999,"5min":0.224,"15min":0.22600000000000001},"min":{"1min":0.14899999999999999,"5min":0.14199999999999999,"15min":0.14199999999999999},"max":{"1min":0.32200000000000001,"5min":0.443,"15min":0.47999999999999998},"last":0.17199999999999999},{"interface":"front","average":{"1min":0.249,"5min":0.26100000000000001,"15min":0.26200000000000001},"min":{"1min":0.20699999999999999,"5min":0.17999999999999999,"15min":0.16},"max":{"1min":0.313,"5min":0.47499999999999998,"15min":0.51000000000000001},"last":0.23699999999999999}]},{"osd":1,"last update":"Wed Oct 18 19:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.224,"5min":0.23599999999999999,"15min":0.23499999999999999},"min":{"1min":0.17999999999999999,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.27900000000000003,"5min":0.44,"15min":0.48899999999999999},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.23899999999999999,"5min":0.247,"15min":0.245},"min":{"1min":0.186,"5min":0.17199999999999999,"15min":0.16900000000000001},"max":{"1min":0.29799999999999999,"5min":0.46100000000000002,"15min":0.48199999999999998},"last":0.20000000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740686,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:11:34 2023","interfaces":[{"interface":"back","average":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.20699999999999999},"min":{"1min":0.156,"5min":0.153,"15min":0.11799999999999999},"max":{"1min":0.36799999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.224,"5min":0.23100000000000001,"15min":0.22800000000000001},"min":{"1min":0.156,"5min":0.156,"15min":0.156},"max":{"1min":0.38300000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.23599999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:11:34 2023","interfaces":[{"interface":"back","average":{"1min":0.23000000000000001,"5min":0.23300000000000001,"15min":0.22800000000000001},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.39500000000000002,"5min":0.44600000000000001,"15min":0.44700000000000001},"last":0.246},{"interface":"front","average":{"1min":0.23699999999999999,"5min":0.249,"15min":0.24399999999999999},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.14399999999999999},"max":{"1min":0.40300000000000002,"5min":0.498,"15min":0.498},"last":0.26900000000000002}]}]},{"osd":1,"up_from":8,"seq":34359740686,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:11:18 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.22,"15min":0.214},"min":{"1min":0.16700000000000001,"5min":0.152,"15min":0.13800000000000001},"max":{"1min":0.42199999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.17399999999999999},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.249,"15min":0.245},"min":{"1min":0.19900000000000001,"5min":0.17599999999999999,"15min":0.17299999999999999},"max":{"1min":0.47399999999999998,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.20100000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:11:18 2023","interfaces":[{"interface":"back","average":{"1min":0.24399999999999999,"5min":0.23899999999999999,"15min":0.23599999999999999},"min":{"1min":0.20100000000000001,"5min":0.17899999999999999,"15min":0.16900000000000001},"max":{"1min":0.42999999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.27900000000000003,"5min":0.26700000000000002,"15min":0.26300000000000001},"min":{"1min":0.22800000000000001,"5min":0.19600000000000001,"15min":0.18099999999999999},"max":{"1min":0.45100000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.22600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:11:57.146 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:57.146 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:57.147 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:57.147 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:57.148 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:57.148 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:57.148 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:57.149 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:57.149 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:57.149 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:11:57.150 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:11:57.670 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:57.667+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:57.975 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:57.972+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:58.717 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:58.714+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:58.934 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:58.931+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:59.712 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:11:59.709+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:11:59.980 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:11:59.977+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:00.667 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:00.664+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:00.956 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:00.953+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:01.682 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:01.679+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:01.935 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:01.932+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:02.691 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:02.688+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:02.946 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:02.943+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:03.644 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:03.641+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:03.956 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:03.953+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:04.612 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:04.609+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:04.926 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:04.923+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:05.602 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:05.599+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:05.937 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:05.934+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:06.626 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:06.623+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:06.919 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:06.916+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:07.648 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:07.645+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:07.882 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:07.879+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:08.686 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:08.683+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:08.852 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:08.849+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:09.701 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:09.698+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:09.862 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:09.859+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:10.665 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:10.662+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:10.884 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:10.881+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:11.674 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:11.671+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:11.878 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:11.875+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:12.719 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:12.716+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:12.848 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:12.845+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:13.687 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:13.684+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:13.841 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:13.838+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:14.680 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:14.677+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:14.847 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:14.844+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:15.654 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:15.651+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:15.888 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:15.885+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:16.644 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:16.640+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:16.864 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:16.861+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:17.151 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:12:17.463 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:12:17.465 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:12:17.475 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5942,"stamp":"2023-10-18T19:12:16.578904+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001623"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740688,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.20499999999999999,"5min":0.224,"15min":0.22600000000000001},"min":{"1min":0.14899999999999999,"5min":0.14199999999999999,"15min":0.14199999999999999},"max":{"1min":0.32200000000000001,"5min":0.443,"15min":0.47999999999999998},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.249,"5min":0.26100000000000001,"15min":0.26200000000000001},"min":{"1min":0.20699999999999999,"5min":0.17999999999999999,"15min":0.16},"max":{"1min":0.313,"5min":0.47499999999999998,"15min":0.51000000000000001},"last":0.28799999999999998}]},{"osd":1,"last update":"Wed Oct 18 19:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.224,"5min":0.23599999999999999,"15min":0.23499999999999999},"min":{"1min":0.17999999999999999,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.27900000000000003,"5min":0.44,"15min":0.48899999999999999},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.23899999999999999,"5min":0.247,"15min":0.245},"min":{"1min":0.186,"5min":0.17199999999999999,"15min":0.16900000000000001},"max":{"1min":0.29799999999999999,"5min":0.46100000000000002,"15min":0.48199999999999998},"last":0.27200000000000002}]}]},{"osd":0,"up_from":8,"seq":34359740690,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:11:34 2023","interfaces":[{"interface":"back","average":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.20699999999999999},"min":{"1min":0.156,"5min":0.153,"15min":0.11799999999999999},"max":{"1min":0.36799999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.17000000000000001},{"interface":"front","average":{"1min":0.224,"5min":0.23100000000000001,"15min":0.22800000000000001},"min":{"1min":0.156,"5min":0.156,"15min":0.156},"max":{"1min":0.38300000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.223}]},{"osd":2,"last update":"Wed Oct 18 19:11:34 2023","interfaces":[{"interface":"back","average":{"1min":0.23000000000000001,"5min":0.23300000000000001,"15min":0.22800000000000001},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.39500000000000002,"5min":0.44600000000000001,"15min":0.44700000000000001},"last":0.20200000000000001},{"interface":"front","average":{"1min":0.23699999999999999,"5min":0.249,"15min":0.24399999999999999},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.14399999999999999},"max":{"1min":0.40300000000000002,"5min":0.498,"15min":0.498},"last":0.20899999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740690,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:11:18 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.22,"15min":0.214},"min":{"1min":0.16700000000000001,"5min":0.152,"15min":0.13800000000000001},"max":{"1min":0.42199999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.249,"15min":0.245},"min":{"1min":0.19900000000000001,"5min":0.17599999999999999,"15min":0.17299999999999999},"max":{"1min":0.47399999999999998,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.28499999999999998}]},{"osd":2,"last update":"Wed Oct 18 19:11:18 2023","interfaces":[{"interface":"back","average":{"1min":0.24399999999999999,"5min":0.23899999999999999,"15min":0.23599999999999999},"min":{"1min":0.20100000000000001,"5min":0.17899999999999999,"15min":0.16900000000000001},"max":{"1min":0.42999999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.27900000000000003,"5min":0.26700000000000002,"15min":0.26300000000000001},"min":{"1min":0.22800000000000001,"5min":0.19600000000000001,"15min":0.18099999999999999},"max":{"1min":0.45100000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.26100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:12:17.477 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:17.478 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:17.478 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:17.478 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:17.479 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:17.479 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:17.480 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:17.480 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:17.480 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:17.481 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:17.481 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:12:17.645 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:17.642+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:17.880 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:17.877+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:18.622 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:18.619+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:18.896 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:18.893+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:19.573 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:19.570+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:19.914 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:19.911+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:20.578 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:20.575+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:20.950 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:20.947+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:21.626 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:21.623+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:21.949 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:21.946+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:22.579 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:22.576+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:22.923 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:22.920+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:23.624 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:23.621+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:23.894 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:23.891+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:24.597 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:24.594+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:24.855 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:24.852+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:25.558 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:25.555+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:25.809 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:25.806+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:26.551 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:26.548+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:26.772 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:26.770+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:27.574 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:27.571+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:27.755 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:27.753+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:28.594 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:28.592+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:28.800 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:28.798+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:29.613 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:29.611+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:29.753 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:29.751+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:30.578 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:30.576+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:30.715 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:30.713+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:31.572 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:31.570+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:31.747 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:31.746+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:32.540 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:32.538+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:32.711 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:32.709+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:33.550 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:33.548+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:33.690 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:33.688+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:34.586 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:34.584+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:34.733 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:34.731+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:35.560 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:35.558+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:35.774 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:35.772+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:36.584 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:36.582+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:36.795 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:36.793+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:37.483 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:12:37.576 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:37.574+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:37.792 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:12:37.794 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:12:37.805 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5952,"stamp":"2023-10-18T19:12:36.582078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001927"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740692,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.20499999999999999,"5min":0.224,"15min":0.22600000000000001},"min":{"1min":0.14899999999999999,"5min":0.14199999999999999,"15min":0.14199999999999999},"max":{"1min":0.32200000000000001,"5min":0.443,"15min":0.47999999999999998},"last":0.20699999999999999},{"interface":"front","average":{"1min":0.249,"5min":0.26100000000000001,"15min":0.26200000000000001},"min":{"1min":0.20699999999999999,"5min":0.17999999999999999,"15min":0.16},"max":{"1min":0.313,"5min":0.47499999999999998,"15min":0.51000000000000001},"last":0.27200000000000002}]},{"osd":1,"last update":"Wed Oct 18 19:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.224,"5min":0.23599999999999999,"15min":0.23499999999999999},"min":{"1min":0.17999999999999999,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.27900000000000003,"5min":0.44,"15min":0.48899999999999999},"last":0.245},{"interface":"front","average":{"1min":0.23899999999999999,"5min":0.247,"15min":0.245},"min":{"1min":0.186,"5min":0.17199999999999999,"15min":0.16900000000000001},"max":{"1min":0.29799999999999999,"5min":0.46100000000000002,"15min":0.48199999999999998},"last":0.28399999999999997}]}]},{"osd":0,"up_from":8,"seq":34359740694,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.20899999999999999,"5min":0.215,"15min":0.20699999999999999},"min":{"1min":0.157,"5min":0.153,"15min":0.11799999999999999},"max":{"1min":0.46300000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.189},{"interface":"front","average":{"1min":0.23000000000000001,"5min":0.23599999999999999,"15min":0.22800000000000001},"min":{"1min":0.17399999999999999,"5min":0.156,"15min":0.156},"max":{"1min":0.41299999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.22700000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.23599999999999999,"5min":0.23699999999999999,"15min":0.22800000000000001},"min":{"1min":0.152,"5min":0.152,"15min":0.152},"max":{"1min":0.42799999999999999,"5min":0.44600000000000001,"15min":0.44700000000000001},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.24299999999999999,"5min":0.249,"15min":0.24399999999999999},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.14399999999999999},"max":{"1min":0.44900000000000001,"5min":0.498,"15min":0.498},"last":0.247}]}]},{"osd":1,"up_from":8,"seq":34359740694,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:12:23 2023","interfaces":[{"interface":"back","average":{"1min":0.20799999999999999,"5min":0.223,"15min":0.214},"min":{"1min":0.17199999999999999,"5min":0.155,"15min":0.13800000000000001},"max":{"1min":0.29399999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.24399999999999999,"5min":0.253,"15min":0.245},"min":{"1min":0.20100000000000001,"5min":0.189,"15min":0.17299999999999999},"max":{"1min":0.36399999999999999,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.29499999999999998}]},{"osd":2,"last update":"Wed Oct 18 19:12:23 2023","interfaces":[{"interface":"back","average":{"1min":0.23000000000000001,"5min":0.24099999999999999,"15min":0.23599999999999999},"min":{"1min":0.19400000000000001,"5min":0.17899999999999999,"15min":0.16900000000000001},"max":{"1min":0.28000000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.27100000000000002,"15min":0.26300000000000001},"min":{"1min":0.224,"5min":0.19800000000000001,"15min":0.18099999999999999},"max":{"1min":0.41799999999999998,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.26900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:12:37.806 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:37.807 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:37.807 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:37.807 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:37.808 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:37.808 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:37.808 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:37.809 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:37.809 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:37.810 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:37.810 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:12:37.841 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:37.839+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:38.586 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:38.584+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:38.831 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:38.829+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:39.552 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:39.551+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:39.837 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:39.835+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:40.587 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:40.585+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:40.802 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:40.801+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:41.621 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:41.619+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:41.851 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:41.849+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:42.618 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:42.616+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:42.808 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:42.806+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:43.574 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:43.572+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:43.761 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:43.759+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:44.591 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:44.589+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:44.721 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:44.720+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:45.551 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:45.549+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:45.760 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:45.758+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:46.518 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:46.515+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:46.715 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:46.712+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:47.526 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:47.523+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:47.696 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:47.693+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:48.479 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:48.477+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:48.652 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:48.650+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:49.480 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:49.477+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:49.665 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:49.662+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:50.499 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:50.496+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:50.637 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:50.634+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:51.546 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:51.543+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:51.632 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:51.629+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:52.568 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:52.565+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:52.627 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:52.624+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:53.581 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:53.578+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:53.634 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:53.631+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:54.579 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:54.576+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:54.669 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:54.666+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:55.559 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:55.556+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:55.709 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:55.706+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:56.594 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:56.591+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:56.700 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:56.697+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:57.585 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:57.582+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:57.655 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:57.652+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:57.812 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:12:58.113 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:12:58.115 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:12:58.125 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5962,"stamp":"2023-10-18T19:12:56.585051+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001883"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740696,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.23599999999999999,"5min":0.224,"15min":0.22600000000000001},"min":{"1min":0.16300000000000001,"5min":0.14899999999999999,"15min":0.14199999999999999},"max":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47999999999999998},"last":0.16700000000000001},{"interface":"front","average":{"1min":0.27000000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"min":{"1min":0.19800000000000001,"5min":0.17999999999999999,"15min":0.16},"max":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.51000000000000001},"last":0.23499999999999999}]},{"osd":1,"last update":"Wed Oct 18 19:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.249,"5min":0.23699999999999999,"15min":0.23599999999999999},"min":{"1min":0.19500000000000001,"5min":0.17999999999999999,"15min":0.17000000000000001},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.48899999999999999},"last":0.19},{"interface":"front","average":{"1min":0.252,"5min":0.246,"15min":0.24399999999999999},"min":{"1min":0.193,"5min":0.186,"15min":0.16900000000000001},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.223}]}]},{"osd":0,"up_from":8,"seq":34359740698,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.20899999999999999,"5min":0.215,"15min":0.20699999999999999},"min":{"1min":0.157,"5min":0.153,"15min":0.11799999999999999},"max":{"1min":0.46300000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.158},{"interface":"front","average":{"1min":0.23000000000000001,"5min":0.23599999999999999,"15min":0.22800000000000001},"min":{"1min":0.17399999999999999,"5min":0.156,"15min":0.156},"max":{"1min":0.41299999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.20300000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.23599999999999999,"5min":0.23699999999999999,"15min":0.22800000000000001},"min":{"1min":0.152,"5min":0.152,"15min":0.152},"max":{"1min":0.42799999999999999,"5min":0.44600000000000001,"15min":0.44700000000000001},"last":0.182},{"interface":"front","average":{"1min":0.24299999999999999,"5min":0.249,"15min":0.24399999999999999},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.14399999999999999},"max":{"1min":0.44900000000000001,"5min":0.498,"15min":0.498},"last":0.193}]}]},{"osd":1,"up_from":8,"seq":34359740698,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:12:23 2023","interfaces":[{"interface":"back","average":{"1min":0.20799999999999999,"5min":0.223,"15min":0.214},"min":{"1min":0.17199999999999999,"5min":0.155,"15min":0.13800000000000001},"max":{"1min":0.29399999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.24399999999999999,"5min":0.253,"15min":0.245},"min":{"1min":0.20100000000000001,"5min":0.189,"15min":0.17299999999999999},"max":{"1min":0.36399999999999999,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.23799999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:12:23 2023","interfaces":[{"interface":"back","average":{"1min":0.23000000000000001,"5min":0.24099999999999999,"15min":0.23599999999999999},"min":{"1min":0.19400000000000001,"5min":0.17899999999999999,"15min":0.16900000000000001},"max":{"1min":0.28000000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.19900000000000001},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.27100000000000002,"15min":0.26300000000000001},"min":{"1min":0.224,"5min":0.19800000000000001,"15min":0.18099999999999999},"max":{"1min":0.41799999999999998,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.26000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:12:58.127 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:58.127 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:58.128 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:58.128 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:58.128 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:58.129 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:58.129 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:58.129 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:58.130 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:58.130 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:12:58.131 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:12:58.599 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:58.595+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:58.646 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:58.643+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:59.649 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:12:59.646+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:12:59.681 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:12:59.678+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:00.636 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:00.633+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:00.651 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:00.648+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:01.659 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:01.656+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:01.697 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:01.694+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:02.690 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:02.687+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:02.718 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:02.715+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:03.709 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:03.706+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:03.750 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:03.747+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:04.748 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:04.745+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:04.778 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:04.775+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:05.728 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:05.725+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:05.825 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:05.823+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:06.720 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:06.717+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:06.849 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:06.846+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:07.689 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:07.686+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:07.800 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:07.797+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:08.647 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:08.644+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:08.831 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:08.828+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:09.672 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:09.669+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:09.855 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:09.852+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:10.708 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:10.705+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:10.889 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:10.887+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:11.753 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:11.750+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:11.919 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:11.916+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:12.796 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:12.793+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:12.946 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:12.943+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:13.843 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:13.840+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:13.951 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:13.948+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:14.872 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:14.869+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:14.987 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:14.984+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:15.900 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:15.897+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:16.036 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:16.033+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:16.947 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:16.944+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:16.988 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:16.985+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:17.957 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:17.954+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:17.963 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:17.961+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:18.132 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:13:18.444 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:13:18.446 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:13:18.457 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5972,"stamp":"2023-10-18T19:13:16.587849+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001673"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740700,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.23599999999999999,"5min":0.224,"15min":0.22600000000000001},"min":{"1min":0.16300000000000001,"5min":0.14899999999999999,"15min":0.14199999999999999},"max":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47999999999999998},"last":0.183},{"interface":"front","average":{"1min":0.27000000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"min":{"1min":0.19800000000000001,"5min":0.17999999999999999,"15min":0.16},"max":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.51000000000000001},"last":0.251}]},{"osd":1,"last update":"Wed Oct 18 19:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.249,"5min":0.23699999999999999,"15min":0.23599999999999999},"min":{"1min":0.19500000000000001,"5min":0.17999999999999999,"15min":0.17000000000000001},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.48899999999999999},"last":0.218},{"interface":"front","average":{"1min":0.252,"5min":0.246,"15min":0.24399999999999999},"min":{"1min":0.193,"5min":0.186,"15min":0.16900000000000001},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.20100000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740702,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.20899999999999999,"5min":0.215,"15min":0.20699999999999999},"min":{"1min":0.157,"5min":0.153,"15min":0.11799999999999999},"max":{"1min":0.46300000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.313},{"interface":"front","average":{"1min":0.23000000000000001,"5min":0.23599999999999999,"15min":0.22800000000000001},"min":{"1min":0.17399999999999999,"5min":0.156,"15min":0.156},"max":{"1min":0.41299999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.36699999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.23599999999999999,"5min":0.23699999999999999,"15min":0.22800000000000001},"min":{"1min":0.152,"5min":0.152,"15min":0.152},"max":{"1min":0.42799999999999999,"5min":0.44600000000000001,"15min":0.44700000000000001},"last":0.377},{"interface":"front","average":{"1min":0.24299999999999999,"5min":0.249,"15min":0.24399999999999999},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.14399999999999999},"max":{"1min":0.44900000000000001,"5min":0.498,"15min":0.498},"last":0.32100000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740702,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:12:23 2023","interfaces":[{"interface":"back","average":{"1min":0.20799999999999999,"5min":0.223,"15min":0.214},"min":{"1min":0.17199999999999999,"5min":0.155,"15min":0.13800000000000001},"max":{"1min":0.29399999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.24399999999999999,"5min":0.253,"15min":0.245},"min":{"1min":0.20100000000000001,"5min":0.189,"15min":0.17299999999999999},"max":{"1min":0.36399999999999999,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.42299999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:12:23 2023","interfaces":[{"interface":"back","average":{"1min":0.23000000000000001,"5min":0.24099999999999999,"15min":0.23599999999999999},"min":{"1min":0.19400000000000001,"5min":0.17899999999999999,"15min":0.16900000000000001},"max":{"1min":0.28000000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.27100000000000002,"15min":0.26300000000000001},"min":{"1min":0.224,"5min":0.19800000000000001,"15min":0.18099999999999999},"max":{"1min":0.41799999999999998,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.44900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:13:18.458 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:13:18.459 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:13:18.459 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:13:18.459 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:13:18.460 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:13:18.460 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:13:18.460 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:13:18.461 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:13:18.461 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:13:18.462 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:13:18.462 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.7 2023-10-18T19:13:18.758 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.7 on osd.1 to deep-scrub 2023-10-18T19:13:18.769 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-10-18T19:13:18.963 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:18.960+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:18.976 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:18.973+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:19.104 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-10-18T19:13:19.115 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.5 2023-10-18T19:13:19.467 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.5 on osd.1 to deep-scrub 2023-10-18T19:13:19.479 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.1 2023-10-18T19:13:19.788 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.1 on osd.0 to deep-scrub 2023-10-18T19:13:19.799 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.2 2023-10-18T19:13:19.946 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:19.944+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:19.965 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:19.963+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:20.092 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.2 on osd.1 to deep-scrub 2023-10-18T19:13:20.103 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-10-18T19:13:20.394 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-10-18T19:13:20.405 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.0 2023-10-18T19:13:20.709 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2023-10-18T19:13:20.721 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.3 2023-10-18T19:13:20.951 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:20.949+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:21.003 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:21.001+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:21.057 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.3 on osd.0 to deep-scrub 2023-10-18T19:13:21.067 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.2 2023-10-18T19:13:21.363 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.2 on osd.0 to deep-scrub 2023-10-18T19:13:21.374 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-10-18T19:13:21.670 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-10-18T19:13:21.680 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:13:21.965 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:21.963+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:21.995 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:21.993+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:22.919 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:22.917+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:23.035 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:23.033+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:23.953 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:23.951+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:24.032 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:24.030+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:24.973 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:24.971+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:25.071 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:25.069+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:25.955 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:25.952+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:26.121 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:26.119+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:26.925 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:26.923+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:27.072 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:27.070+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:27.877 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:27.875+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:28.121 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:28.119+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:28.872 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:28.870+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:29.149 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:29.147+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:29.866 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:29.864+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:30.105 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:30.103+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:30.834 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:30.832+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:31.085 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:31.083+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:31.821 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:31.819+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:32.053 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:32.051+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:32.793 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:32.791+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:33.085 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:33.083+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:33.763 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:33.761+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:34.046 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:34.044+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:34.782 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:34.780+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:35.077 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:35.075+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:35.774 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:35.772+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:36.032 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:36.030+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:36.810 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:36.808+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:37.055 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:37.052+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:37.859 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:37.856+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:38.059 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:38.056+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:38.843 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:38.840+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:39.078 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:39.075+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:39.843 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:39.840+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:40.042 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:40.039+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:40.832 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:40.829+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:41.053 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:41.050+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:41.682 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:13:41.803 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:41.800+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:41.976 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:13:41.978 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:13:41.988 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5984,"stamp":"2023-10-18T19:13:40.591553+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001831"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740705,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.23599999999999999,"5min":0.224,"15min":0.22600000000000001},"min":{"1min":0.16300000000000001,"5min":0.14899999999999999,"15min":0.14199999999999999},"max":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47999999999999998},"last":0.186},{"interface":"front","average":{"1min":0.27000000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"min":{"1min":0.19800000000000001,"5min":0.17999999999999999,"15min":0.16},"max":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.51000000000000001},"last":0.254}]},{"osd":1,"last update":"Wed Oct 18 19:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.249,"5min":0.23699999999999999,"15min":0.23599999999999999},"min":{"1min":0.19500000000000001,"5min":0.17999999999999999,"15min":0.17000000000000001},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.48899999999999999},"last":0.22},{"interface":"front","average":{"1min":0.252,"5min":0.246,"15min":0.24399999999999999},"min":{"1min":0.193,"5min":0.186,"15min":0.16900000000000001},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.23200000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740706,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.20899999999999999,"5min":0.215,"15min":0.20699999999999999},"min":{"1min":0.157,"5min":0.153,"15min":0.11799999999999999},"max":{"1min":0.46300000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.17699999999999999},{"interface":"front","average":{"1min":0.23000000000000001,"5min":0.23599999999999999,"15min":0.22800000000000001},"min":{"1min":0.17399999999999999,"5min":0.156,"15min":0.156},"max":{"1min":0.41299999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.221}]},{"osd":2,"last update":"Wed Oct 18 19:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.23599999999999999,"5min":0.23699999999999999,"15min":0.22800000000000001},"min":{"1min":0.152,"5min":0.152,"15min":0.152},"max":{"1min":0.42799999999999999,"5min":0.44600000000000001,"15min":0.44700000000000001},"last":0.21199999999999999},{"interface":"front","average":{"1min":0.24299999999999999,"5min":0.249,"15min":0.24399999999999999},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.14399999999999999},"max":{"1min":0.44900000000000001,"5min":0.498,"15min":0.498},"last":0.24199999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740706,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.23999999999999999,"5min":0.224,"15min":0.216},"min":{"1min":0.188,"5min":0.155,"15min":0.14899999999999999},"max":{"1min":0.39200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.17999999999999999},{"interface":"front","average":{"1min":0.27300000000000002,"5min":0.25600000000000001,"15min":0.247},"min":{"1min":0.19700000000000001,"5min":0.189,"15min":0.17299999999999999},"max":{"1min":0.42299999999999999,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.20799999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.252,"5min":0.24399999999999999,"15min":0.23699999999999999},"min":{"1min":0.187,"5min":0.17899999999999999,"15min":0.16900000000000001},"max":{"1min":0.40699999999999997,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.28699999999999998,"5min":0.27400000000000002,"15min":0.26500000000000001},"min":{"1min":0.22900000000000001,"5min":0.19800000000000001,"15min":0.18099999999999999},"max":{"1min":0.44900000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.23100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:13:41.990 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:13:41.991 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:13:41.991 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:13:41.991 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:13:41.992 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:13:41.992 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:13:41.993 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:13:41.993 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:13:41.993 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:13:41.994 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:13:41.994 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:13:42.087 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:42.084+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:42.809 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:42.806+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:43.059 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:43.056+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:43.775 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:43.771+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:44.048 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:44.045+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:44.791 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:44.787+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:45.010 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:45.007+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:45.750 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:45.748+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:46.047 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:46.044+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:46.713 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:46.710+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:47.014 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:47.011+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:47.734 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:47.731+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:48.016 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:48.013+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:48.741 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:48.738+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:49.001 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:48.998+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:49.706 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:49.703+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:49.977 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:49.974+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:50.689 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:50.686+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:50.957 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:50.954+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:51.652 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:51.649+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:51.929 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:51.926+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:52.620 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:52.616+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:52.949 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:52.946+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:53.589 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:53.586+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:53.975 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:53.972+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:54.559 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:54.556+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:54.937 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:54.934+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:55.585 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:55.582+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:55.912 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:55.909+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:56.574 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:56.571+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:56.951 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:56.948+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:57.529 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:57.526+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:57.978 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:57.974+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:58.537 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:58.534+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:58.970 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:58.967+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:59.576 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:13:59.573+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:13:59.922 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:13:59.919+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:00.614 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:00.610+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:00.932 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:00.929+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:01.649 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:01.647+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:01.908 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:01.905+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:01.996 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:14:02.317 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:14:02.319 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:14:02.331 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5994,"stamp":"2023-10-18T19:14:00.594720+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001897"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740709,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.221,"5min":0.22700000000000001,"15min":0.22600000000000001},"min":{"1min":0.158,"5min":0.14899999999999999,"15min":0.14199999999999999},"max":{"1min":0.33900000000000002,"5min":0.47599999999999998,"15min":0.47999999999999998},"last":0.20599999999999999},{"interface":"front","average":{"1min":0.26000000000000001,"5min":0.26200000000000001,"15min":0.26100000000000001},"min":{"1min":0.20000000000000001,"5min":0.189,"15min":0.16},"max":{"1min":0.33100000000000002,"5min":0.46899999999999997,"15min":0.51000000000000001},"last":0.27100000000000002}]},{"osd":1,"last update":"Wed Oct 18 19:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.23599999999999999,"15min":0.23499999999999999},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17000000000000001},"max":{"1min":0.29999999999999999,"5min":0.44700000000000001,"15min":0.48899999999999999},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.23899999999999999,"5min":0.245,"15min":0.24399999999999999},"min":{"1min":0.20100000000000001,"5min":0.186,"15min":0.16900000000000001},"max":{"1min":0.31900000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.253}]}]},{"osd":0,"up_from":8,"seq":34359740710,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.24299999999999999,"5min":0.217,"15min":0.20899999999999999},"min":{"1min":0.158,"5min":0.153,"15min":0.11799999999999999},"max":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.17399999999999999},{"interface":"front","average":{"1min":0.25900000000000001,"5min":0.23799999999999999,"15min":0.23000000000000001},"min":{"1min":0.19500000000000001,"5min":0.156,"15min":0.156},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.48399999999999999},"last":0.20999999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.26200000000000001,"5min":0.23999999999999999,"15min":0.23000000000000001},"min":{"1min":0.182,"5min":0.152,"15min":0.152},"max":{"1min":0.443,"5min":0.443,"15min":0.44700000000000001},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.27400000000000002,"5min":0.251,"15min":0.246},"min":{"1min":0.193,"5min":0.16600000000000001,"15min":0.14399999999999999},"max":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.498},"last":0.23699999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740710,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.23999999999999999,"5min":0.224,"15min":0.216},"min":{"1min":0.188,"5min":0.155,"15min":0.14899999999999999},"max":{"1min":0.39200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.20799999999999999},{"interface":"front","average":{"1min":0.27300000000000002,"5min":0.25600000000000001,"15min":0.247},"min":{"1min":0.19700000000000001,"5min":0.189,"15min":0.17299999999999999},"max":{"1min":0.42299999999999999,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.25800000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.252,"5min":0.24399999999999999,"15min":0.23699999999999999},"min":{"1min":0.187,"5min":0.17899999999999999,"15min":0.16900000000000001},"max":{"1min":0.40699999999999997,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.217},{"interface":"front","average":{"1min":0.28699999999999998,"5min":0.27400000000000002,"15min":0.26500000000000001},"min":{"1min":0.22900000000000001,"5min":0.19800000000000001,"15min":0.18099999999999999},"max":{"1min":0.44900000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.23799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:14:02.332 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:02.333 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:02.333 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:02.334 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:02.334 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:02.334 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:02.335 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:02.335 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:02.335 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:02.336 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:02.336 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:14:02.619 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:02.616+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:02.926 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:02.923+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:03.578 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:03.575+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:03.896 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:03.893+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:04.585 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:04.582+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:04.847 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:04.844+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:05.601 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:05.598+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:05.806 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:05.803+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:06.643 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:06.640+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:06.789 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:06.786+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:07.656 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:07.652+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:07.802 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:07.800+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:08.655 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:08.652+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:08.783 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:08.780+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:09.705 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:09.702+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:09.819 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:09.816+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:10.666 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:10.664+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:10.857 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:10.855+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:11.674 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:11.672+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:11.860 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:11.858+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:12.679 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:12.677+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:12.892 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:12.890+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:13.667 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:13.665+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:13.884 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:13.882+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:14.640 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:14.638+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:14.908 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:14.906+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:15.630 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:15.628+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:15.903 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:15.901+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:16.611 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:16.609+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:16.881 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:16.879+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:17.583 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:17.581+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:17.908 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:17.906+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:18.567 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:18.565+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:18.911 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:18.909+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:19.533 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:19.531+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:19.897 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:19.895+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:20.522 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:20.520+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:20.936 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:20.934+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:21.559 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:21.557+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:21.898 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:21.896+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:22.337 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:14:22.518 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:22.516+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:22.667 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:14:22.669 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:14:22.680 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":6005,"stamp":"2023-10-18T19:14:22.598204+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001960"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740714,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.221,"5min":0.22700000000000001,"15min":0.22600000000000001},"min":{"1min":0.158,"5min":0.14899999999999999,"15min":0.14199999999999999},"max":{"1min":0.33900000000000002,"5min":0.47599999999999998,"15min":0.47999999999999998},"last":0.376},{"interface":"front","average":{"1min":0.26000000000000001,"5min":0.26200000000000001,"15min":0.26100000000000001},"min":{"1min":0.20000000000000001,"5min":0.189,"15min":0.16},"max":{"1min":0.33100000000000002,"5min":0.46899999999999997,"15min":0.51000000000000001},"last":0.36799999999999999}]},{"osd":1,"last update":"Wed Oct 18 19:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.23599999999999999,"15min":0.23499999999999999},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17000000000000001},"max":{"1min":0.29999999999999999,"5min":0.44700000000000001,"15min":0.48899999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.23899999999999999,"5min":0.245,"15min":0.24399999999999999},"min":{"1min":0.20100000000000001,"5min":0.186,"15min":0.16900000000000001},"max":{"1min":0.31900000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.34899999999999998}]}]},{"osd":0,"up_from":8,"seq":34359740715,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.24299999999999999,"5min":0.217,"15min":0.20899999999999999},"min":{"1min":0.158,"5min":0.153,"15min":0.11799999999999999},"max":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.20100000000000001},{"interface":"front","average":{"1min":0.25900000000000001,"5min":0.23799999999999999,"15min":0.23000000000000001},"min":{"1min":0.19500000000000001,"5min":0.156,"15min":0.156},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.48399999999999999},"last":0.219}]},{"osd":2,"last update":"Wed Oct 18 19:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.26200000000000001,"5min":0.23999999999999999,"15min":0.23000000000000001},"min":{"1min":0.182,"5min":0.152,"15min":0.152},"max":{"1min":0.443,"5min":0.443,"15min":0.44700000000000001},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.27400000000000002,"5min":0.251,"15min":0.246},"min":{"1min":0.193,"5min":0.16600000000000001,"15min":0.14399999999999999},"max":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.498},"last":0.24299999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740715,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.23999999999999999,"5min":0.224,"15min":0.216},"min":{"1min":0.188,"5min":0.155,"15min":0.14899999999999999},"max":{"1min":0.39200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.19},{"interface":"front","average":{"1min":0.27300000000000002,"5min":0.25600000000000001,"15min":0.247},"min":{"1min":0.19700000000000001,"5min":0.189,"15min":0.17299999999999999},"max":{"1min":0.42299999999999999,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.22600000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.252,"5min":0.24399999999999999,"15min":0.23699999999999999},"min":{"1min":0.187,"5min":0.17899999999999999,"15min":0.16900000000000001},"max":{"1min":0.40699999999999997,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.218},{"interface":"front","average":{"1min":0.28699999999999998,"5min":0.27400000000000002,"15min":0.26500000000000001},"min":{"1min":0.22900000000000001,"5min":0.19800000000000001,"15min":0.18099999999999999},"max":{"1min":0.44900000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.25600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:14:22.681 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:22.682 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:22.682 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:22.683 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:22.683 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:22.683 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:22.684 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:22.684 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:22.685 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:22.685 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:22.685 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:14:22.942 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:22.940+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:23.562 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:23.560+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:23.915 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:23.913+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:24.575 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:24.573+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:24.906 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:24.904+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:25.619 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:25.617+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:25.897 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:25.895+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:26.655 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:26.653+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:26.895 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:26.893+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:27.685 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:27.682+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:27.891 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:27.888+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:28.707 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:28.704+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:28.845 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:28.842+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:29.671 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:29.668+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:29.867 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:29.864+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:30.688 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:30.685+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:30.823 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:30.820+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:31.701 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:31.698+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:31.801 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:31.798+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:32.741 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:32.738+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:32.779 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:32.776+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:33.732 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:33.729+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:33.757 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:33.754+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:34.734 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:34.731+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:34.771 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:34.768+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:35.722 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:35.719+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:35.818 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:35.815+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:36.727 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:36.724+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:36.775 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:36.772+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:37.696 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:37.693+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:37.822 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:37.819+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:38.735 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:38.732+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:38.795 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:38.793+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:39.706 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:39.703+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:39.804 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:39.801+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:40.677 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:40.674+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:40.837 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:40.834+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:41.673 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:41.670+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:41.805 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:41.802+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:42.672 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:42.669+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:42.687 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:14:42.795 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:42.792+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:42.998 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:14:43.000 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:14:43.010 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":6015,"stamp":"2023-10-18T19:14:42.601290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001883"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740718,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.22600000000000001,"5min":0.222,"15min":0.22500000000000001},"min":{"1min":0.16700000000000001,"5min":0.14899999999999999,"15min":0.14199999999999999},"max":{"1min":0.376,"5min":0.47599999999999998,"15min":0.47999999999999998},"last":0.251},{"interface":"front","average":{"1min":0.26800000000000002,"5min":0.26300000000000001,"15min":0.26100000000000001},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.16600000000000001},"max":{"1min":0.36799999999999999,"5min":0.46899999999999997,"15min":0.47499999999999998},"last":0.24199999999999999}]},{"osd":1,"last update":"Wed Oct 18 19:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.23999999999999999,"5min":0.23599999999999999,"15min":0.23499999999999999},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.33600000000000002,"5min":0.44700000000000001,"15min":0.48599999999999999},"last":0.20300000000000001},{"interface":"front","average":{"1min":0.25800000000000001,"5min":0.248,"15min":0.24399999999999999},"min":{"1min":0.21099999999999999,"5min":0.186,"15min":0.17199999999999999},"max":{"1min":0.34899999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.23300000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740719,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:14:39 2023","interfaces":[{"interface":"back","average":{"1min":0.217,"5min":0.217,"15min":0.20799999999999999},"min":{"1min":0.14399999999999999,"5min":0.14399999999999999,"15min":0.11799999999999999},"max":{"1min":0.40300000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.222},{"interface":"front","average":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23000000000000001},"min":{"1min":0.186,"5min":0.156,"15min":0.156},"max":{"1min":0.42199999999999999,"5min":0.46000000000000002,"15min":0.48399999999999999},"last":0.188}]},{"osd":2,"last update":"Wed Oct 18 19:14:39 2023","interfaces":[{"interface":"back","average":{"1min":0.24299999999999999,"5min":0.24099999999999999,"15min":0.23000000000000001},"min":{"1min":0.17199999999999999,"5min":0.152,"15min":0.152},"max":{"1min":0.438,"5min":0.443,"15min":0.44700000000000001},"last":0.19500000000000001},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.252,"15min":0.246},"min":{"1min":0.19700000000000001,"5min":0.16600000000000001,"15min":0.14399999999999999},"max":{"1min":0.44500000000000001,"5min":0.46600000000000003,"15min":0.498},"last":0.20699999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740719,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.215,"5min":0.224,"15min":0.217},"min":{"1min":0.17699999999999999,"5min":0.16700000000000001,"15min":0.14899999999999999},"max":{"1min":0.30599999999999999,"5min":0.42199999999999999,"15min":0.46200000000000002},"last":0.16300000000000001},{"interface":"front","average":{"1min":0.23899999999999999,"5min":0.253,"15min":0.248},"min":{"1min":0.188,"5min":0.188,"15min":0.17299999999999999},"max":{"1min":0.28599999999999998,"5min":0.47399999999999998,"15min":0.51000000000000001},"last":0.19800000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.221,"5min":0.23899999999999999,"15min":0.23599999999999999},"min":{"1min":0.187,"5min":0.186,"15min":0.16900000000000001},"max":{"1min":0.25,"5min":0.42999999999999999,"15min":0.44700000000000001},"last":0.187},{"interface":"front","average":{"1min":0.254,"5min":0.27200000000000002,"15min":0.26500000000000001},"min":{"1min":0.22600000000000001,"5min":0.222,"15min":0.18099999999999999},"max":{"1min":0.28399999999999997,"5min":0.47099999999999997,"15min":0.48899999999999999},"last":0.224}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:14:43.012 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:43.013 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:43.013 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:43.013 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:43.014 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:43.014 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:43.014 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:43.015 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:43.015 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:43.015 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:14:43.016 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:14:43.663 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:43.660+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:43.763 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:43.760+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:44.623 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:44.620+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:44.761 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:44.758+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:45.621 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:45.618+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:45.761 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:45.758+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:46.672 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:46.669+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:46.749 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:46.746+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:47.682 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:47.679+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:47.701 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:47.698+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:48.638 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:48.635+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:48.730 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:48.727+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:49.622 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:49.619+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:49.736 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:49.733+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:50.643 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:50.640+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:50.700 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:50.697+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:51.651 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:51.648+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:51.689 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:51.686+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:52.630 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:52.627+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:52.651 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:52.648+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:53.613 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:53.610+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:53.639 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:53.636+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:54.647 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:54.644+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:54.656 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:54.654+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:55.671 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:55.668+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:55.673 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:55.670+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:56.696 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:56.693+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:56.707 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:56.704+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:57.660 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:57.657+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:57.666 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:57.663+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:58.626 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:58.623+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:58.649 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:58.646+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:59.655 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:14:59.652+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:14:59.699 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:14:59.697+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:00.613 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:00.611+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:00.748 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:00.746+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:01.571 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:01.569+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:01.789 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:01.787+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:02.534 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:02.532+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:02.796 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:02.794+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:03.017 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:15:03.307 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:15:03.309 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:15:03.321 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":6025,"stamp":"2023-10-18T19:15:02.604574+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001620"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740722,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.22600000000000001,"5min":0.222,"15min":0.22500000000000001},"min":{"1min":0.16700000000000001,"5min":0.14899999999999999,"15min":0.14199999999999999},"max":{"1min":0.376,"5min":0.47599999999999998,"15min":0.47999999999999998},"last":0.17699999999999999},{"interface":"front","average":{"1min":0.26800000000000002,"5min":0.26300000000000001,"15min":0.26100000000000001},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.16600000000000001},"max":{"1min":0.36799999999999999,"5min":0.46899999999999997,"15min":0.47499999999999998},"last":0.24199999999999999}]},{"osd":1,"last update":"Wed Oct 18 19:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.23999999999999999,"5min":0.23599999999999999,"15min":0.23499999999999999},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.33600000000000002,"5min":0.44700000000000001,"15min":0.48599999999999999},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.25800000000000001,"5min":0.248,"15min":0.24399999999999999},"min":{"1min":0.21099999999999999,"5min":0.186,"15min":0.17199999999999999},"max":{"1min":0.34899999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.215}]}]},{"osd":0,"up_from":8,"seq":34359740723,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:14:39 2023","interfaces":[{"interface":"back","average":{"1min":0.217,"5min":0.217,"15min":0.20799999999999999},"min":{"1min":0.14399999999999999,"5min":0.14399999999999999,"15min":0.11799999999999999},"max":{"1min":0.40300000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.215},{"interface":"front","average":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23000000000000001},"min":{"1min":0.186,"5min":0.156,"15min":0.156},"max":{"1min":0.42199999999999999,"5min":0.46000000000000002,"15min":0.48399999999999999},"last":0.184}]},{"osd":2,"last update":"Wed Oct 18 19:14:39 2023","interfaces":[{"interface":"back","average":{"1min":0.24299999999999999,"5min":0.24099999999999999,"15min":0.23000000000000001},"min":{"1min":0.17199999999999999,"5min":0.152,"15min":0.152},"max":{"1min":0.438,"5min":0.443,"15min":0.44700000000000001},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.252,"15min":0.246},"min":{"1min":0.19700000000000001,"5min":0.16600000000000001,"15min":0.14399999999999999},"max":{"1min":0.44500000000000001,"5min":0.46600000000000003,"15min":0.498},"last":0.20799999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740723,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.215,"5min":0.224,"15min":0.217},"min":{"1min":0.17699999999999999,"5min":0.16700000000000001,"15min":0.14899999999999999},"max":{"1min":0.30599999999999999,"5min":0.42199999999999999,"15min":0.46200000000000002},"last":0.214},{"interface":"front","average":{"1min":0.23899999999999999,"5min":0.253,"15min":0.248},"min":{"1min":0.188,"5min":0.188,"15min":0.17299999999999999},"max":{"1min":0.28599999999999998,"5min":0.47399999999999998,"15min":0.51000000000000001},"last":0.254}]},{"osd":2,"last update":"Wed Oct 18 19:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.221,"5min":0.23899999999999999,"15min":0.23599999999999999},"min":{"1min":0.187,"5min":0.186,"15min":0.16900000000000001},"max":{"1min":0.25,"5min":0.42999999999999999,"15min":0.44700000000000001},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.254,"5min":0.27200000000000002,"15min":0.26500000000000001},"min":{"1min":0.22600000000000001,"5min":0.222,"15min":0.18099999999999999},"max":{"1min":0.28399999999999997,"5min":0.47099999999999997,"15min":0.48899999999999999},"last":0.28799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:15:03.323 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:03.323 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:03.323 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:03.324 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:03.324 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:03.324 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:03.325 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:03.325 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:03.325 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:03.326 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:03.326 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:15:03.513 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:03.511+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:03.813 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:03.811+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:04.561 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:04.559+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:04.774 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:04.772+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:05.530 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:05.528+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:05.793 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:05.791+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:06.573 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:06.571+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:06.832 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:06.830+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:07.567 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:07.565+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:07.827 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:07.825+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:08.520 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:08.518+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:08.848 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:08.846+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:09.486 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:09.484+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:09.892 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:09.890+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:10.493 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:10.491+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:10.892 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:10.890+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:11.484 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:11.482+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:11.927 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:11.925+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:12.442 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:12.440+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:12.899 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:12.897+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:13.465 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:13.463+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:13.939 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:13.937+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:14.441 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:14.439+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:14.963 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:14.961+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:15.464 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:15.462+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:15.963 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:15.961+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:16.462 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:16.460+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:16.920 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:16.918+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:17.483 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:17.481+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:17.874 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:17.871+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:18.522 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:18.519+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:18.879 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:18.876+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:19.496 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:19.493+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:19.858 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:19.856+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:20.482 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:20.479+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:20.870 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:20.867+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:21.433 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:21.430+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:21.915 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:21.912+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:22.427 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:22.424+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:22.905 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:22.902+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:23.330 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:15:23.398 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:23.395+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:23.632 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:15:23.634 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:15:23.646 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":6035,"stamp":"2023-10-18T19:15:22.607612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001894"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740726,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.22600000000000001,"5min":0.222,"15min":0.22500000000000001},"min":{"1min":0.16700000000000001,"5min":0.14899999999999999,"15min":0.14199999999999999},"max":{"1min":0.376,"5min":0.47599999999999998,"15min":0.47999999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.26800000000000002,"5min":0.26300000000000001,"15min":0.26100000000000001},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.16600000000000001},"max":{"1min":0.36799999999999999,"5min":0.46899999999999997,"15min":0.47499999999999998},"last":0.45500000000000002}]},{"osd":1,"last update":"Wed Oct 18 19:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.23999999999999999,"5min":0.23599999999999999,"15min":0.23499999999999999},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.33600000000000002,"5min":0.44700000000000001,"15min":0.48599999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.25800000000000001,"5min":0.248,"15min":0.24399999999999999},"min":{"1min":0.21099999999999999,"5min":0.186,"15min":0.17199999999999999},"max":{"1min":0.34899999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.442}]}]},{"osd":0,"up_from":8,"seq":34359740727,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:14:39 2023","interfaces":[{"interface":"back","average":{"1min":0.217,"5min":0.217,"15min":0.20799999999999999},"min":{"1min":0.14399999999999999,"5min":0.14399999999999999,"15min":0.11799999999999999},"max":{"1min":0.40300000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.215},{"interface":"front","average":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23000000000000001},"min":{"1min":0.186,"5min":0.156,"15min":0.156},"max":{"1min":0.42199999999999999,"5min":0.46000000000000002,"15min":0.48399999999999999},"last":0.19700000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:14:39 2023","interfaces":[{"interface":"back","average":{"1min":0.24299999999999999,"5min":0.24099999999999999,"15min":0.23000000000000001},"min":{"1min":0.17199999999999999,"5min":0.152,"15min":0.152},"max":{"1min":0.438,"5min":0.443,"15min":0.44700000000000001},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.252,"15min":0.246},"min":{"1min":0.19700000000000001,"5min":0.16600000000000001,"15min":0.14399999999999999},"max":{"1min":0.44500000000000001,"5min":0.46600000000000003,"15min":0.498},"last":0.23899999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740727,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.215,"5min":0.224,"15min":0.217},"min":{"1min":0.17699999999999999,"5min":0.16700000000000001,"15min":0.14899999999999999},"max":{"1min":0.30599999999999999,"5min":0.42199999999999999,"15min":0.46200000000000002},"last":0.19600000000000001},{"interface":"front","average":{"1min":0.23899999999999999,"5min":0.253,"15min":0.248},"min":{"1min":0.188,"5min":0.188,"15min":0.17299999999999999},"max":{"1min":0.28599999999999998,"5min":0.47399999999999998,"15min":0.51000000000000001},"last":0.23200000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.221,"5min":0.23899999999999999,"15min":0.23599999999999999},"min":{"1min":0.187,"5min":0.186,"15min":0.16900000000000001},"max":{"1min":0.25,"5min":0.42999999999999999,"15min":0.44700000000000001},"last":0.224},{"interface":"front","average":{"1min":0.254,"5min":0.27200000000000002,"15min":0.26500000000000001},"min":{"1min":0.22600000000000001,"5min":0.222,"15min":0.18099999999999999},"max":{"1min":0.28399999999999997,"5min":0.47099999999999997,"15min":0.48899999999999999},"last":0.26700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:15:23.647 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:23.648 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:23.648 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:23.648 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:23.649 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:23.649 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:23.650 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:23.650 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:23.650 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:23.651 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:23.651 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.7 2023-10-18T19:15:23.859 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:23.856+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:24.003 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.7 on osd.1 to deep-scrub 2023-10-18T19:15:24.015 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-10-18T19:15:24.338 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-10-18T19:15:24.349 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.5 2023-10-18T19:15:24.421 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:24.418+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:24.669 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.5 on osd.1 to deep-scrub 2023-10-18T19:15:24.680 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.1 2023-10-18T19:15:24.842 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:24.839+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:24.978 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.1 on osd.0 to deep-scrub 2023-10-18T19:15:24.991 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.2 2023-10-18T19:15:25.303 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.2 on osd.1 to deep-scrub 2023-10-18T19:15:25.315 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-10-18T19:15:25.438 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:25.435+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:25.620 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-10-18T19:15:25.630 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.0 2023-10-18T19:15:25.872 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:25.869+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:25.941 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2023-10-18T19:15:25.953 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.3 2023-10-18T19:15:26.264 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.3 on osd.0 to deep-scrub 2023-10-18T19:15:26.276 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.2 2023-10-18T19:15:26.489 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:26.486+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:26.576 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.2 on osd.0 to deep-scrub 2023-10-18T19:15:26.589 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-10-18T19:15:26.879 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:26.876+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:26.882 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-10-18T19:15:26.893 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:15:27.525 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:27.522+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:27.876 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:27.873+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:28.511 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:28.508+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:28.871 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:28.868+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:29.522 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:29.519+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:29.857 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:29.854+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:30.520 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:30.517+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:30.872 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:30.869+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:31.569 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:31.566+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:31.911 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:31.908+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:32.562 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:32.559+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:32.941 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:32.938+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:33.539 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:33.536+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:33.953 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:33.950+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:34.504 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:34.501+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:34.913 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:34.910+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:35.502 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:35.499+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:35.930 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:35.927+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:36.493 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:36.490+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:36.934 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:36.932+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:37.521 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:37.518+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:37.929 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:37.926+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:38.549 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:38.546+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:38.973 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:38.970+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:39.584 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:39.581+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:39.930 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:39.927+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:40.559 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:40.556+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:40.903 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:40.900+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:41.595 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:41.592+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:41.929 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:41.926+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:42.577 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:42.574+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:42.909 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:42.907+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:43.615 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:43.612+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:43.949 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:43.946+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:44.579 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:44.576+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:44.919 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:44.916+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:45.570 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:45.568+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:45.947 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:45.945+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:46.614 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:46.611+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:46.894 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:15:46.946 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:46.943+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:47.184 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:15:47.186 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:15:47.198 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":6047,"stamp":"2023-10-18T19:15:46.611714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002146"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740730,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.22600000000000001,"5min":0.222,"15min":0.22500000000000001},"min":{"1min":0.16700000000000001,"5min":0.14899999999999999,"15min":0.14199999999999999},"max":{"1min":0.376,"5min":0.47599999999999998,"15min":0.47999999999999998},"last":0.17399999999999999},{"interface":"front","average":{"1min":0.26800000000000002,"5min":0.26300000000000001,"15min":0.26100000000000001},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.16600000000000001},"max":{"1min":0.36799999999999999,"5min":0.46899999999999997,"15min":0.47499999999999998},"last":0.26000000000000001}]},{"osd":1,"last update":"Wed Oct 18 19:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.23999999999999999,"5min":0.23599999999999999,"15min":0.23499999999999999},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.33600000000000002,"5min":0.44700000000000001,"15min":0.48599999999999999},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.25800000000000001,"5min":0.248,"15min":0.24399999999999999},"min":{"1min":0.21099999999999999,"5min":0.186,"15min":0.17199999999999999},"max":{"1min":0.34899999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.23499999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740732,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.23200000000000001,"5min":0.222,"15min":0.20899999999999999},"min":{"1min":0.17299999999999999,"5min":0.14399999999999999,"15min":0.11799999999999999},"max":{"1min":0.41199999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.23599999999999999,"5min":0.23699999999999999,"15min":0.23000000000000001},"min":{"1min":0.16,"5min":0.156,"15min":0.156},"max":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.48399999999999999},"last":0.35199999999999998}]},{"osd":2,"last update":"Wed Oct 18 19:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.25,"5min":0.24399999999999999,"15min":0.23100000000000001},"min":{"1min":0.192,"5min":0.152,"15min":0.152},"max":{"1min":0.433,"5min":0.443,"15min":0.44600000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.27600000000000002,"5min":0.25700000000000001,"15min":0.248},"min":{"1min":0.20699999999999999,"5min":0.16600000000000001,"15min":0.14399999999999999},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.39200000000000002}]}]},{"osd":1,"up_from":8,"seq":34359740732,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:15:31 2023","interfaces":[{"interface":"back","average":{"1min":0.22600000000000001,"5min":0.224,"15min":0.218},"min":{"1min":0.16300000000000001,"5min":0.16300000000000001,"15min":0.152},"max":{"1min":0.38300000000000001,"5min":0.42199999999999999,"15min":0.46200000000000002},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.253,"15min":0.249},"min":{"1min":0.189,"5min":0.188,"15min":0.17599999999999999},"max":{"1min":0.44400000000000001,"5min":0.47399999999999998,"15min":0.51000000000000001},"last":0.25600000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:15:31 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.23599999999999999,"15min":0.23699999999999999},"min":{"1min":0.187,"5min":0.187,"15min":0.16900000000000001},"max":{"1min":0.40600000000000003,"5min":0.42999999999999999,"15min":0.44700000000000001},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.27200000000000002,"15min":0.26600000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.18099999999999999},"max":{"1min":0.443,"5min":0.45100000000000001,"15min":0.48899999999999999},"last":0.224}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:15:47.200 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:47.200 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:47.201 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:47.201 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:47.201 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:47.202 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:47.202 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:47.202 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:47.203 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:47.203 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:15:47.203 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:15:47.614 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:47.612+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:47.974 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:47.971+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:48.582 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:48.580+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:48.962 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:48.959+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:49.589 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:49.586+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:50.007 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:50.005+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:50.589 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:50.586+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:51.001 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:50.998+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:51.625 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:51.623+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:52.009 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:52.007+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:52.599 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:52.597+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:52.985 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:52.982+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:53.600 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:53.598+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:54.018 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:54.016+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:54.588 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:54.586+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:55.027 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:55.026+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:55.569 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:55.567+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:56.038 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:56.036+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:56.541 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:56.538+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:57.019 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:57.017+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:57.529 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:57.527+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:58.051 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:58.049+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:58.481 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:58.479+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:59.060 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:15:59.058+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:15:59.486 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:15:59.484+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:00.022 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:00.020+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:00.505 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:00.503+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:00.977 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:00.976+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:01.520 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:01.518+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:01.952 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:01.950+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:02.523 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:02.521+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:02.961 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:02.959+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:03.557 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:03.555+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:03.936 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:03.934+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:04.608 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:04.606+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:04.924 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:04.922+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:05.602 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:05.600+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:05.880 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:05.878+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:06.638 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:06.636+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:06.862 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:06.860+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:07.205 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:16:07.512 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:16:07.514 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:16:07.525 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":6057,"stamp":"2023-10-18T19:16:06.615130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001688"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740734,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.20699999999999999,"5min":0.219,"15min":0.223},"min":{"1min":0.159,"5min":0.14899999999999999,"15min":0.14199999999999999},"max":{"1min":0.38200000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"min":{"1min":0.20300000000000001,"5min":0.17999999999999999,"15min":0.16600000000000001},"max":{"1min":0.45500000000000002,"5min":0.46899999999999997,"15min":0.47499999999999998},"last":0.30499999999999999}]},{"osd":1,"last update":"Wed Oct 18 19:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.23599999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"min":{"1min":0.20100000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.41699999999999998,"5min":0.44700000000000001,"15min":0.48599999999999999},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.23999999999999999,"5min":0.245,"15min":0.24299999999999999},"min":{"1min":0.19,"5min":0.186,"15min":0.17199999999999999},"max":{"1min":0.442,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.25700000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740736,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.23200000000000001,"5min":0.222,"15min":0.20899999999999999},"min":{"1min":0.17299999999999999,"5min":0.14399999999999999,"15min":0.11799999999999999},"max":{"1min":0.41199999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.17799999999999999},{"interface":"front","average":{"1min":0.23599999999999999,"5min":0.23699999999999999,"15min":0.23000000000000001},"min":{"1min":0.16,"5min":0.156,"15min":0.156},"max":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.48399999999999999},"last":0.217}]},{"osd":2,"last update":"Wed Oct 18 19:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.25,"5min":0.24399999999999999,"15min":0.23100000000000001},"min":{"1min":0.192,"5min":0.152,"15min":0.152},"max":{"1min":0.433,"5min":0.443,"15min":0.44600000000000001},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.27600000000000002,"5min":0.25700000000000001,"15min":0.248},"min":{"1min":0.20699999999999999,"5min":0.16600000000000001,"15min":0.14399999999999999},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.25}]}]},{"osd":1,"up_from":8,"seq":34359740736,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:15:31 2023","interfaces":[{"interface":"back","average":{"1min":0.22600000000000001,"5min":0.224,"15min":0.218},"min":{"1min":0.16300000000000001,"5min":0.16300000000000001,"15min":0.152},"max":{"1min":0.38300000000000001,"5min":0.42199999999999999,"15min":0.46200000000000002},"last":0.20699999999999999},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.253,"15min":0.249},"min":{"1min":0.189,"5min":0.188,"15min":0.17599999999999999},"max":{"1min":0.44400000000000001,"5min":0.47399999999999998,"15min":0.51000000000000001},"last":0.27800000000000002}]},{"osd":2,"last update":"Wed Oct 18 19:15:31 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.23599999999999999,"15min":0.23699999999999999},"min":{"1min":0.187,"5min":0.187,"15min":0.16900000000000001},"max":{"1min":0.40600000000000003,"5min":0.42999999999999999,"15min":0.44700000000000001},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.27200000000000002,"15min":0.26600000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.18099999999999999},"max":{"1min":0.443,"5min":0.45100000000000001,"15min":0.48899999999999999},"last":0.248}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:16:07.526 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:07.527 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:07.527 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:07.527 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:07.528 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:07.528 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:07.528 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:07.529 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:07.529 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:07.529 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:07.530 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:16:07.605 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:07.603+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:07.902 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:07.900+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:08.642 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:08.639+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:08.910 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:08.907+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:09.669 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:09.666+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:09.939 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:09.936+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:10.681 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:10.677+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:10.938 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:10.935+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:11.713 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:11.710+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:11.938 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:11.935+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:12.703 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:12.700+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:12.986 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:12.983+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:13.691 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:13.688+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:14.034 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:14.031+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:14.725 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:14.722+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:15.039 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:15.036+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:15.754 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:15.751+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:16.073 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:16.070+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:16.721 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:16.718+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:17.034 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:17.031+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:17.748 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:17.745+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:18.001 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:17.998+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:18.733 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:18.731+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:18.971 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:18.968+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:19.778 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:19.775+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:19.973 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:19.970+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:20.809 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:20.807+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:20.950 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:20.947+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:21.856 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:21.853+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:21.904 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:21.901+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:22.863 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:22.860+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:22.942 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:22.940+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:23.838 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:23.835+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:23.913 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:23.910+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:24.864 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:24.861+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:24.895 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:24.892+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:25.874 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:25.871+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:25.927 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:25.924+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:26.886 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:26.883+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:26.887 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:26.883+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:27.531 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:16:27.832 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:16:27.834 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:16:27.843 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:27.840+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:27.846 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":6067,"stamp":"2023-10-18T19:16:26.618207+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001990"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740738,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.20699999999999999,"5min":0.219,"15min":0.223},"min":{"1min":0.159,"5min":0.14899999999999999,"15min":0.14199999999999999},"max":{"1min":0.38200000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"last":0.17799999999999999},{"interface":"front","average":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"min":{"1min":0.20300000000000001,"5min":0.17999999999999999,"15min":0.16600000000000001},"max":{"1min":0.45500000000000002,"5min":0.46899999999999997,"15min":0.47499999999999998},"last":0.24299999999999999}]},{"osd":1,"last update":"Wed Oct 18 19:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.23599999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"min":{"1min":0.20100000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.41699999999999998,"5min":0.44700000000000001,"15min":0.48599999999999999},"last":0.20799999999999999},{"interface":"front","average":{"1min":0.23999999999999999,"5min":0.245,"15min":0.24299999999999999},"min":{"1min":0.19,"5min":0.186,"15min":0.17199999999999999},"max":{"1min":0.442,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.22900000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740740,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.23200000000000001,"5min":0.222,"15min":0.20899999999999999},"min":{"1min":0.17299999999999999,"5min":0.14399999999999999,"15min":0.11799999999999999},"max":{"1min":0.41199999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.19400000000000001},{"interface":"front","average":{"1min":0.23599999999999999,"5min":0.23699999999999999,"15min":0.23000000000000001},"min":{"1min":0.16,"5min":0.156,"15min":0.156},"max":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.48399999999999999},"last":0.26600000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.25,"5min":0.24399999999999999,"15min":0.23100000000000001},"min":{"1min":0.192,"5min":0.152,"15min":0.152},"max":{"1min":0.433,"5min":0.443,"15min":0.44600000000000001},"last":0.218},{"interface":"front","average":{"1min":0.27600000000000002,"5min":0.25700000000000001,"15min":0.248},"min":{"1min":0.20699999999999999,"5min":0.16600000000000001,"15min":0.14399999999999999},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.23400000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740740,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:15:31 2023","interfaces":[{"interface":"back","average":{"1min":0.22600000000000001,"5min":0.224,"15min":0.218},"min":{"1min":0.16300000000000001,"5min":0.16300000000000001,"15min":0.152},"max":{"1min":0.38300000000000001,"5min":0.42199999999999999,"15min":0.46200000000000002},"last":0.161},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.253,"15min":0.249},"min":{"1min":0.189,"5min":0.188,"15min":0.17599999999999999},"max":{"1min":0.44400000000000001,"5min":0.47399999999999998,"15min":0.51000000000000001},"last":0.19}]},{"osd":2,"last update":"Wed Oct 18 19:15:31 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.23599999999999999,"15min":0.23699999999999999},"min":{"1min":0.187,"5min":0.187,"15min":0.16900000000000001},"max":{"1min":0.40600000000000003,"5min":0.42999999999999999,"15min":0.44700000000000001},"last":0.17899999999999999},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.27200000000000002,"15min":0.26600000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.18099999999999999},"max":{"1min":0.443,"5min":0.45100000000000001,"15min":0.48899999999999999},"last":0.215}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:16:27.848 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:27.848 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:27.849 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:27.849 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:27.850 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:27.850 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:27.851 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:27.851 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:27.851 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:27.852 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:27.852 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:16:27.853 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:27.847+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:28.807 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:28.804+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:28.817 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:28.814+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:29.759 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:29.756+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:29.783 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:29.780+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:30.760 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:30.757+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:30.822 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:30.819+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:31.747 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:31.744+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:31.865 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:31.862+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:32.777 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:32.774+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:32.842 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:32.839+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:33.826 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:33.823+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:33.874 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:33.871+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:34.796 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:34.793+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:34.894 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:34.892+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:35.792 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:35.789+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:35.877 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:35.874+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:36.783 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:36.758+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:36.830 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:36.827+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:37.735 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:37.733+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:37.834 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:37.831+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:38.785 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:38.782+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:38.881 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:38.878+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:39.776 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:39.773+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:39.858 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:39.855+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:40.743 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:40.741+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:40.881 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:40.878+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:41.771 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:41.769+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:41.903 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:41.901+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:42.760 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:42.758+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:42.863 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:42.861+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:43.732 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:43.730+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:43.890 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:43.888+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:44.696 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:44.695+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:44.937 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:44.934+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:45.744 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:45.742+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:45.895 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:45.893+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:46.789 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:46.787+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:46.873 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:46.871+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:47.819 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:47.817+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:47.852 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:47.850+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:47.853 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:16:48.154 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:16:48.156 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:16:48.167 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":6077,"stamp":"2023-10-18T19:16:46.620940+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001633"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740742,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.20699999999999999,"5min":0.219,"15min":0.223},"min":{"1min":0.159,"5min":0.14899999999999999,"15min":0.14199999999999999},"max":{"1min":0.38200000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"min":{"1min":0.20300000000000001,"5min":0.17999999999999999,"15min":0.16600000000000001},"max":{"1min":0.45500000000000002,"5min":0.46899999999999997,"15min":0.47499999999999998},"last":0.29199999999999998}]},{"osd":1,"last update":"Wed Oct 18 19:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.23599999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"min":{"1min":0.20100000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.41699999999999998,"5min":0.44700000000000001,"15min":0.48599999999999999},"last":0.255},{"interface":"front","average":{"1min":0.23999999999999999,"5min":0.245,"15min":0.24299999999999999},"min":{"1min":0.19,"5min":0.186,"15min":0.17199999999999999},"max":{"1min":0.442,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.27700000000000002}]}]},{"osd":0,"up_from":8,"seq":34359740744,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.217,"5min":0.223,"15min":0.21099999999999999},"min":{"1min":0.158,"5min":0.14399999999999999,"15min":0.11799999999999999},"max":{"1min":0.42099999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.19500000000000001},{"interface":"front","average":{"1min":0.22600000000000001,"5min":0.23699999999999999,"15min":0.23100000000000001},"min":{"1min":0.16600000000000001,"5min":0.16,"15min":0.156},"max":{"1min":0.35199999999999998,"5min":0.47299999999999998,"15min":0.48399999999999999},"last":0.223}]},{"osd":2,"last update":"Wed Oct 18 19:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.23000000000000001,"5min":0.24399999999999999,"15min":0.23300000000000001},"min":{"1min":0.191,"5min":0.152,"15min":0.152},"max":{"1min":0.36599999999999999,"5min":0.443,"15min":0.44600000000000001},"last":0.251},{"interface":"front","average":{"1min":0.25800000000000001,"5min":0.26100000000000001,"15min":0.25},"min":{"1min":0.19900000000000001,"5min":0.16600000000000001,"15min":0.14399999999999999},"max":{"1min":0.39200000000000002,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.25900000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740744,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:16:35 2023","interfaces":[{"interface":"back","average":{"1min":0.223,"5min":0.222,"15min":0.219},"min":{"1min":0.161,"5min":0.161,"15min":0.152},"max":{"1min":0.311,"5min":0.39200000000000002,"15min":0.46200000000000002},"last":0.18099999999999999},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.253,"15min":0.249},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.17599999999999999},"max":{"1min":0.34799999999999998,"5min":0.44400000000000001,"15min":0.51000000000000001},"last":0.22700000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:16:35 2023","interfaces":[{"interface":"back","average":{"1min":0.23200000000000001,"5min":0.23400000000000001,"15min":0.23699999999999999},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.16900000000000001},"max":{"1min":0.35799999999999998,"5min":0.40699999999999997,"15min":0.44700000000000001},"last":0.16300000000000001},{"interface":"front","average":{"1min":0.26100000000000001,"5min":0.26900000000000002,"15min":0.26600000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.18099999999999999},"max":{"1min":0.379,"5min":0.44900000000000001,"15min":0.48899999999999999},"last":0.20399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:16:48.168 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:48.169 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:48.169 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:48.170 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:48.170 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:48.170 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:48.171 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:48.171 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:48.172 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:48.172 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:16:48.172 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:16:48.780 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:48.778+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:48.856 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:48.854+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:49.768 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:49.766+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:49.879 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:49.878+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:50.751 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:50.749+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:50.846 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:50.844+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:51.735 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:51.733+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:51.836 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:51.834+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:52.689 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:52.686+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:52.803 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:52.801+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:53.730 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:53.728+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:53.838 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:53.836+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:54.770 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:54.768+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:54.854 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:54.852+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:55.756 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:55.754+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:55.900 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:55.898+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:56.792 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:56.790+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:56.895 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:56.893+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:57.782 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:57.780+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:57.898 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:57.896+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:58.778 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:58.774+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:58.937 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:58.934+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:59.734 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:16:59.731+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:16:59.957 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:16:59.954+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:00.707 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:00.704+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:00.997 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:00.994+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:01.694 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:01.691+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:02.004 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:02.001+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:02.727 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:02.724+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:03.030 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:03.027+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:03.738 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:03.735+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:04.000 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:03.997+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:04.694 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:04.691+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:04.968 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:04.965+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:05.728 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:05.725+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:06.016 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:06.013+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:06.726 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:06.723+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:07.025 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:07.022+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:07.720 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:07.717+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:08.075 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:08.072+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:08.174 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:17:08.521 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:17:08.523 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:17:08.535 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":6087,"stamp":"2023-10-18T19:17:06.623692+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001657"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740746,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.223,"15min":0.223},"min":{"1min":0.16,"5min":0.158,"15min":0.14199999999999999},"max":{"1min":0.371,"5min":0.47599999999999998,"15min":0.47599999999999998},"last":0.188},{"interface":"front","average":{"1min":0.27300000000000002,"5min":0.26600000000000001,"15min":0.26200000000000001},"min":{"1min":0.217,"5min":0.17999999999999999,"15min":0.16600000000000001},"max":{"1min":0.38100000000000001,"5min":0.46899999999999997,"15min":0.47499999999999998},"last":0.26200000000000001}]},{"osd":1,"last update":"Wed Oct 18 19:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.23300000000000001,"5min":0.23699999999999999,"15min":0.23499999999999999},"min":{"1min":0.185,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.33800000000000002,"5min":0.44700000000000001,"15min":0.48599999999999999},"last":0.221},{"interface":"front","average":{"1min":0.248,"5min":0.247,"15min":0.24399999999999999},"min":{"1min":0.191,"5min":0.19,"15min":0.17199999999999999},"max":{"1min":0.35399999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.22900000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740748,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.217,"5min":0.223,"15min":0.21099999999999999},"min":{"1min":0.158,"5min":0.14399999999999999,"15min":0.11799999999999999},"max":{"1min":0.42099999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.22600000000000001,"5min":0.23699999999999999,"15min":0.23100000000000001},"min":{"1min":0.16600000000000001,"5min":0.16,"15min":0.156},"max":{"1min":0.35199999999999998,"5min":0.47299999999999998,"15min":0.48399999999999999},"last":0.247}]},{"osd":2,"last update":"Wed Oct 18 19:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.23000000000000001,"5min":0.24399999999999999,"15min":0.23300000000000001},"min":{"1min":0.191,"5min":0.152,"15min":0.152},"max":{"1min":0.36599999999999999,"5min":0.443,"15min":0.44600000000000001},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.25800000000000001,"5min":0.26100000000000001,"15min":0.25},"min":{"1min":0.19900000000000001,"5min":0.16600000000000001,"15min":0.14399999999999999},"max":{"1min":0.39200000000000002,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.28299999999999997}]}]},{"osd":1,"up_from":8,"seq":34359740748,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:16:35 2023","interfaces":[{"interface":"back","average":{"1min":0.223,"5min":0.222,"15min":0.219},"min":{"1min":0.161,"5min":0.161,"15min":0.152},"max":{"1min":0.311,"5min":0.39200000000000002,"15min":0.46200000000000002},"last":0.185},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.253,"15min":0.249},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.17599999999999999},"max":{"1min":0.34799999999999998,"5min":0.44400000000000001,"15min":0.51000000000000001},"last":0.214}]},{"osd":2,"last update":"Wed Oct 18 19:16:35 2023","interfaces":[{"interface":"back","average":{"1min":0.23200000000000001,"5min":0.23400000000000001,"15min":0.23699999999999999},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.16900000000000001},"max":{"1min":0.35799999999999998,"5min":0.40699999999999997,"15min":0.44700000000000001},"last":0.19500000000000001},{"interface":"front","average":{"1min":0.26100000000000001,"5min":0.26900000000000002,"15min":0.26600000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.18099999999999999},"max":{"1min":0.379,"5min":0.44900000000000001,"15min":0.48899999999999999},"last":0.23599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:17:08.537 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:08.537 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:08.538 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:08.538 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:08.538 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:08.539 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:08.539 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:08.540 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:08.540 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:08.541 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:08.541 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:17:08.749 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:08.746+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:09.074 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:09.071+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:09.757 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:09.754+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:10.056 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:10.053+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:10.770 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:10.767+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:11.052 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:11.049+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:11.739 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:11.736+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:12.095 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:12.092+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:12.728 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:12.725+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:13.117 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:13.115+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:13.728 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:13.725+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:14.118 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:14.116+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:14.684 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:14.681+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:15.072 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:15.069+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:15.647 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:15.644+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:16.067 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:16.064+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:16.661 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:16.658+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:17.079 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:17.076+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:17.645 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:17.642+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:18.103 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:18.100+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:18.643 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:18.639+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:19.060 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:19.057+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:19.633 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:19.630+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:20.045 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:20.042+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:20.617 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:20.614+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:21.088 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:21.085+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:21.616 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:21.613+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:22.062 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:22.059+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:22.623 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:22.620+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:23.028 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:23.025+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:23.603 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:23.600+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:23.994 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:23.991+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:24.632 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:24.629+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:24.948 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:24.945+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:25.601 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:25.598+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:25.965 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:25.963+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:26.652 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:26.649+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:26.976 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:26.973+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:27.643 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:27.640+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:27.979 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:27.976+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:28.542 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:17:28.616 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:28.613+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:28.836 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:17:28.838 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:17:28.850 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":6098,"stamp":"2023-10-18T19:17:28.626754+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001643"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740751,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.223,"15min":0.223},"min":{"1min":0.16,"5min":0.158,"15min":0.14199999999999999},"max":{"1min":0.371,"5min":0.47599999999999998,"15min":0.47599999999999998},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.27300000000000002,"5min":0.26600000000000001,"15min":0.26200000000000001},"min":{"1min":0.217,"5min":0.17999999999999999,"15min":0.16600000000000001},"max":{"1min":0.38100000000000001,"5min":0.46899999999999997,"15min":0.47499999999999998},"last":0.23899999999999999}]},{"osd":1,"last update":"Wed Oct 18 19:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.23300000000000001,"5min":0.23699999999999999,"15min":0.23499999999999999},"min":{"1min":0.185,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.33800000000000002,"5min":0.44700000000000001,"15min":0.48599999999999999},"last":0.189},{"interface":"front","average":{"1min":0.248,"5min":0.247,"15min":0.24399999999999999},"min":{"1min":0.191,"5min":0.19,"15min":0.17199999999999999},"max":{"1min":0.35399999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.22}]}]},{"osd":0,"up_from":8,"seq":34359740752,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.217,"5min":0.223,"15min":0.21099999999999999},"min":{"1min":0.158,"5min":0.14399999999999999,"15min":0.11799999999999999},"max":{"1min":0.42099999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.157},{"interface":"front","average":{"1min":0.22600000000000001,"5min":0.23699999999999999,"15min":0.23100000000000001},"min":{"1min":0.16600000000000001,"5min":0.16,"15min":0.156},"max":{"1min":0.35199999999999998,"5min":0.47299999999999998,"15min":0.48399999999999999},"last":0.16800000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.23000000000000001,"5min":0.24399999999999999,"15min":0.23300000000000001},"min":{"1min":0.191,"5min":0.152,"15min":0.152},"max":{"1min":0.36599999999999999,"5min":0.443,"15min":0.44600000000000001},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.25800000000000001,"5min":0.26100000000000001,"15min":0.25},"min":{"1min":0.19900000000000001,"5min":0.16600000000000001,"15min":0.14399999999999999},"max":{"1min":0.39200000000000002,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.20200000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740752,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:16:35 2023","interfaces":[{"interface":"back","average":{"1min":0.223,"5min":0.222,"15min":0.219},"min":{"1min":0.161,"5min":0.161,"15min":0.152},"max":{"1min":0.311,"5min":0.39200000000000002,"15min":0.46200000000000002},"last":0.20599999999999999},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.253,"15min":0.249},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.17599999999999999},"max":{"1min":0.34799999999999998,"5min":0.44400000000000001,"15min":0.51000000000000001},"last":0.27700000000000002}]},{"osd":2,"last update":"Wed Oct 18 19:16:35 2023","interfaces":[{"interface":"back","average":{"1min":0.23200000000000001,"5min":0.23400000000000001,"15min":0.23699999999999999},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.16900000000000001},"max":{"1min":0.35799999999999998,"5min":0.40699999999999997,"15min":0.44700000000000001},"last":0.19},{"interface":"front","average":{"1min":0.26100000000000001,"5min":0.26900000000000002,"15min":0.26600000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.18099999999999999},"max":{"1min":0.379,"5min":0.44900000000000001,"15min":0.48899999999999999},"last":0.24299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:17:28.851 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:28.852 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:28.852 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:28.853 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:28.853 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:28.853 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:28.854 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:28.854 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:28.855 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:28.855 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:28.855 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.7 2023-10-18T19:17:28.992 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:28.989+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:29.189 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.7 on osd.1 to deep-scrub 2023-10-18T19:17:29.199 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-10-18T19:17:29.539 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-10-18T19:17:29.552 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.5 2023-10-18T19:17:29.586 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:29.584+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:29.902 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.5 on osd.1 to deep-scrub 2023-10-18T19:17:29.914 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.1 2023-10-18T19:17:30.013 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:30.010+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:30.254 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.1 on osd.0 to deep-scrub 2023-10-18T19:17:30.266 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.2 2023-10-18T19:17:30.617 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.2 on osd.1 to deep-scrub 2023-10-18T19:17:30.629 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-10-18T19:17:30.662 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:30.629+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:30.958 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-10-18T19:17:30.968 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.0 2023-10-18T19:17:31.052 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:31.049+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:31.283 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2023-10-18T19:17:31.294 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.3 2023-10-18T19:17:31.613 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.3 on osd.0 to deep-scrub 2023-10-18T19:17:31.625 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.2 2023-10-18T19:17:31.631 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:31.629+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:31.958 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.2 on osd.0 to deep-scrub 2023-10-18T19:17:31.970 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-10-18T19:17:32.012 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:32.010+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:32.362 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-10-18T19:17:32.373 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:17:32.596 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:32.594+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:33.017 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:33.015+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:33.617 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:33.615+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:34.009 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:34.007+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:34.663 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:34.661+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:35.048 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:35.046+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:35.672 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:35.670+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:36.063 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:36.061+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:36.713 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:36.711+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:37.088 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:37.087+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:37.696 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:37.694+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:38.104 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:38.102+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:38.652 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:38.650+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:39.070 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:39.068+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:39.721 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:39.679+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:40.070 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:40.068+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:40.633 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:40.631+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:41.112 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:41.110+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:41.596 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:41.594+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:42.075 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:42.073+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:42.638 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:42.635+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:43.096 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:43.094+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:43.624 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:43.622+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:44.119 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:44.117+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:44.632 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:44.630+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:45.114 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:45.112+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:45.600 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:45.598+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:46.105 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:46.103+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:46.607 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:46.605+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:47.122 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:47.120+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:47.613 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:47.611+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:48.141 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:48.139+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:48.603 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:48.601+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:49.122 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:49.119+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:49.624 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:49.621+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:50.134 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:50.132+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:50.603 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:50.600+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:51.088 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:51.085+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:51.566 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:51.564+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:52.133 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:52.130+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:52.375 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:17:52.579 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:52.576+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:52.685 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:17:52.688 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:17:52.699 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":6110,"stamp":"2023-10-18T19:17:52.630539+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001976"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740756,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.20300000000000001,"5min":0.216,"15min":0.221},"min":{"1min":0.14599999999999999,"5min":0.14599999999999999,"15min":0.14199999999999999},"max":{"1min":0.254,"5min":0.38200000000000001,"15min":0.47599999999999998},"last":0.25},{"interface":"front","average":{"1min":0.247,"5min":0.26100000000000001,"15min":0.26200000000000001},"min":{"1min":0.19600000000000001,"5min":0.17999999999999999,"15min":0.16600000000000001},"max":{"1min":0.30499999999999999,"5min":0.45500000000000002,"15min":0.47499999999999998},"last":0.33800000000000002}]},{"osd":1,"last update":"Wed Oct 18 19:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.19700000000000001,"5min":0.22600000000000001,"15min":0.23300000000000001},"min":{"1min":0.157,"5min":0.157,"15min":0.157},"max":{"1min":0.221,"5min":0.41699999999999998,"15min":0.48599999999999999},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.22,"5min":0.24099999999999999,"15min":0.24299999999999999},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.17199999999999999},"max":{"1min":0.27200000000000002,"5min":0.442,"15min":0.48699999999999999},"last":0.32900000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740757,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.193,"5min":0.22,"15min":0.20999999999999999},"min":{"1min":0.13700000000000001,"5min":0.13700000000000001,"15min":0.11799999999999999},"max":{"1min":0.249,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.184},{"interface":"front","average":{"1min":0.219,"5min":0.23499999999999999,"15min":0.23100000000000001},"min":{"1min":0.16800000000000001,"5min":0.16,"15min":0.156},"max":{"1min":0.27100000000000002,"5min":0.47299999999999998,"15min":0.48399999999999999},"last":0.20999999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.216,"5min":0.23999999999999999,"15min":0.23300000000000001},"min":{"1min":0.17499999999999999,"5min":0.17199999999999999,"15min":0.152},"max":{"1min":0.254,"5min":0.443,"15min":0.44600000000000001},"last":0.19500000000000001},{"interface":"front","average":{"1min":0.245,"5min":0.26200000000000001,"15min":0.25},"min":{"1min":0.19700000000000001,"5min":0.193,"15min":0.14399999999999999},"max":{"1min":0.29299999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.24199999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740757,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.21099999999999999,"5min":0.223,"15min":0.218},"min":{"1min":0.158,"5min":0.158,"15min":0.152},"max":{"1min":0.34599999999999997,"5min":0.39200000000000002,"15min":0.46200000000000002},"last":0.153},{"interface":"front","average":{"1min":0.248,"5min":0.254,"15min":0.249},"min":{"1min":0.191,"5min":0.17999999999999999,"15min":0.17599999999999999},"max":{"1min":0.379,"5min":0.44400000000000001,"15min":0.51000000000000001},"last":0.191}]},{"osd":2,"last update":"Wed Oct 18 19:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.20799999999999999,"5min":0.23000000000000001,"15min":0.23400000000000001},"min":{"1min":0.154,"5min":0.154,"15min":0.154},"max":{"1min":0.37,"5min":0.40699999999999997,"15min":0.44700000000000001},"last":0.20399999999999999},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.26700000000000002,"15min":0.26500000000000001},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.184},"max":{"1min":0.40699999999999997,"5min":0.44900000000000001,"15min":0.48899999999999999},"last":0.222}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:17:52.700 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:52.701 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:52.701 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:52.701 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:52.702 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:52.702 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:52.703 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:52.703 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:52.703 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:52.704 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:17:52.704 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:17:53.125 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:53.122+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:53.626 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:53.623+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:54.159 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:54.156+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:54.644 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:54.641+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:55.196 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:55.193+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:55.614 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:55.611+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:56.164 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:56.161+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:56.572 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:56.569+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:57.156 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:57.153+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:57.555 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:57.553+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:58.130 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:58.127+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:58.521 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:58.518+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:59.163 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:17:59.161+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:17:59.498 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:17:59.495+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:00.126 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:00.123+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:00.462 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:00.459+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:01.080 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:01.077+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:01.414 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:01.411+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:02.070 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:02.067+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:02.414 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:02.411+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:03.094 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:03.091+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:03.449 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:03.446+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:04.104 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:04.101+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:04.404 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:04.401+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:05.109 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:05.106+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:05.446 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:05.443+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:06.106 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:06.103+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:06.448 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:06.445+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:07.064 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:07.061+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:07.448 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:07.445+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:08.081 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:08.078+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:08.422 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:08.419+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:09.107 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:09.104+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:09.397 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:09.394+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:10.118 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:10.115+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:10.362 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:10.359+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:11.134 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:11.131+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:11.376 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:11.373+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:12.170 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:12.167+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:12.356 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:12.353+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:12.705 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:18:13.162 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:13.159+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:13.208 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:18:13.210 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:18:13.221 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":6120,"stamp":"2023-10-18T19:18:12.633669+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001784"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740760,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.20300000000000001,"5min":0.216,"15min":0.221},"min":{"1min":0.14599999999999999,"5min":0.14599999999999999,"15min":0.14199999999999999},"max":{"1min":0.254,"5min":0.38200000000000001,"15min":0.47599999999999998},"last":0.16900000000000001},{"interface":"front","average":{"1min":0.247,"5min":0.26100000000000001,"15min":0.26200000000000001},"min":{"1min":0.19600000000000001,"5min":0.17999999999999999,"15min":0.16600000000000001},"max":{"1min":0.30499999999999999,"5min":0.45500000000000002,"15min":0.47499999999999998},"last":0.23400000000000001}]},{"osd":1,"last update":"Wed Oct 18 19:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.19700000000000001,"5min":0.22600000000000001,"15min":0.23300000000000001},"min":{"1min":0.157,"5min":0.157,"15min":0.157},"max":{"1min":0.221,"5min":0.41699999999999998,"15min":0.48599999999999999},"last":0.20799999999999999},{"interface":"front","average":{"1min":0.22,"5min":0.24099999999999999,"15min":0.24299999999999999},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.17199999999999999},"max":{"1min":0.27200000000000002,"5min":0.442,"15min":0.48699999999999999},"last":0.22}]}]},{"osd":0,"up_from":8,"seq":34359740761,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.193,"5min":0.22,"15min":0.20999999999999999},"min":{"1min":0.13700000000000001,"5min":0.13700000000000001,"15min":0.11799999999999999},"max":{"1min":0.249,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.159},{"interface":"front","average":{"1min":0.219,"5min":0.23499999999999999,"15min":0.23100000000000001},"min":{"1min":0.16800000000000001,"5min":0.16,"15min":0.156},"max":{"1min":0.27100000000000002,"5min":0.47299999999999998,"15min":0.48399999999999999},"last":0.20699999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.216,"5min":0.23999999999999999,"15min":0.23300000000000001},"min":{"1min":0.17499999999999999,"5min":0.17199999999999999,"15min":0.152},"max":{"1min":0.254,"5min":0.443,"15min":0.44600000000000001},"last":0.19700000000000001},{"interface":"front","average":{"1min":0.245,"5min":0.26200000000000001,"15min":0.25},"min":{"1min":0.19700000000000001,"5min":0.193,"15min":0.14399999999999999},"max":{"1min":0.29299999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.24099999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740761,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.21099999999999999,"5min":0.223,"15min":0.218},"min":{"1min":0.158,"5min":0.158,"15min":0.152},"max":{"1min":0.34599999999999997,"5min":0.39200000000000002,"15min":0.46200000000000002},"last":0.19500000000000001},{"interface":"front","average":{"1min":0.248,"5min":0.254,"15min":0.249},"min":{"1min":0.191,"5min":0.17999999999999999,"15min":0.17599999999999999},"max":{"1min":0.379,"5min":0.44400000000000001,"15min":0.51000000000000001},"last":0.26200000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.20799999999999999,"5min":0.23000000000000001,"15min":0.23400000000000001},"min":{"1min":0.154,"5min":0.154,"15min":0.154},"max":{"1min":0.37,"5min":0.40699999999999997,"15min":0.44700000000000001},"last":0.20599999999999999},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.26700000000000002,"15min":0.26500000000000001},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.184},"max":{"1min":0.40699999999999997,"5min":0.44900000000000001,"15min":0.48899999999999999},"last":0.23000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:18:13.222 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:13.223 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:13.223 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:13.224 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:13.224 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:13.224 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:13.225 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:13.225 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:13.226 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:13.226 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:13.226 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:18:13.384 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:13.381+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:14.120 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:14.117+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:14.349 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:14.346+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:15.140 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:15.137+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:15.340 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:15.337+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:16.138 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:16.135+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:16.376 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:16.374+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:17.164 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:17.161+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:17.381 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:17.378+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:18.202 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:18.199+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:18.393 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:18.390+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:19.245 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:19.242+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:19.435 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:19.432+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:20.234 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:20.231+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:20.450 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:20.447+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:21.186 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:21.183+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:21.490 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:21.487+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:22.222 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:22.220+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:22.476 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:22.474+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:23.259 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:23.257+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:23.436 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:23.434+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:24.270 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:24.268+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:24.433 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:24.431+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:25.255 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:25.253+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:25.459 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:25.457+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:26.241 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:26.239+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:26.494 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:26.492+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:27.282 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:27.280+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:27.458 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:27.456+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:28.251 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:28.249+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:28.428 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:28.426+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:29.280 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:29.278+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:29.464 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:29.462+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:30.236 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:30.234+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:30.455 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:30.453+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:31.272 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:31.270+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:31.412 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:31.410+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:32.232 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:32.230+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:32.444 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:32.442+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:33.227 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:18:33.265 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:33.260+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:33.419 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:33.417+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:33.537 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:18:33.539 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:18:33.551 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":6130,"stamp":"2023-10-18T19:18:32.636641+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001922"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740764,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.20300000000000001,"5min":0.216,"15min":0.221},"min":{"1min":0.14599999999999999,"5min":0.14599999999999999,"15min":0.14199999999999999},"max":{"1min":0.254,"5min":0.38200000000000001,"15min":0.47599999999999998},"last":0.16900000000000001},{"interface":"front","average":{"1min":0.247,"5min":0.26100000000000001,"15min":0.26200000000000001},"min":{"1min":0.19600000000000001,"5min":0.17999999999999999,"15min":0.16600000000000001},"max":{"1min":0.30499999999999999,"5min":0.45500000000000002,"15min":0.47499999999999998},"last":0.22700000000000001}]},{"osd":1,"last update":"Wed Oct 18 19:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.19700000000000001,"5min":0.22600000000000001,"15min":0.23300000000000001},"min":{"1min":0.157,"5min":0.157,"15min":0.157},"max":{"1min":0.221,"5min":0.41699999999999998,"15min":0.48599999999999999},"last":0.20100000000000001},{"interface":"front","average":{"1min":0.22,"5min":0.24099999999999999,"15min":0.24299999999999999},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.17199999999999999},"max":{"1min":0.27200000000000002,"5min":0.442,"15min":0.48699999999999999},"last":0.23899999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740765,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.193,"5min":0.22,"15min":0.20999999999999999},"min":{"1min":0.13700000000000001,"5min":0.13700000000000001,"15min":0.11799999999999999},"max":{"1min":0.249,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.22600000000000001},{"interface":"front","average":{"1min":0.219,"5min":0.23499999999999999,"15min":0.23100000000000001},"min":{"1min":0.16800000000000001,"5min":0.16,"15min":0.156},"max":{"1min":0.27100000000000002,"5min":0.47299999999999998,"15min":0.48399999999999999},"last":0.26200000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.216,"5min":0.23999999999999999,"15min":0.23300000000000001},"min":{"1min":0.17499999999999999,"5min":0.17199999999999999,"15min":0.152},"max":{"1min":0.254,"5min":0.443,"15min":0.44600000000000001},"last":0.25},{"interface":"front","average":{"1min":0.245,"5min":0.26200000000000001,"15min":0.25},"min":{"1min":0.19700000000000001,"5min":0.193,"15min":0.14399999999999999},"max":{"1min":0.29299999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.216}]}]},{"osd":1,"up_from":8,"seq":34359740765,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.21099999999999999,"5min":0.223,"15min":0.218},"min":{"1min":0.158,"5min":0.158,"15min":0.152},"max":{"1min":0.34599999999999997,"5min":0.39200000000000002,"15min":0.46200000000000002},"last":0.183},{"interface":"front","average":{"1min":0.248,"5min":0.254,"15min":0.249},"min":{"1min":0.191,"5min":0.17999999999999999,"15min":0.17599999999999999},"max":{"1min":0.379,"5min":0.44400000000000001,"15min":0.51000000000000001},"last":0.247}]},{"osd":2,"last update":"Wed Oct 18 19:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.20799999999999999,"5min":0.23000000000000001,"15min":0.23400000000000001},"min":{"1min":0.154,"5min":0.154,"15min":0.154},"max":{"1min":0.37,"5min":0.40699999999999997,"15min":0.44700000000000001},"last":0.20000000000000001},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.26700000000000002,"15min":0.26500000000000001},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.184},"max":{"1min":0.40699999999999997,"5min":0.44900000000000001,"15min":0.48899999999999999},"last":0.218}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:18:33.552 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:33.553 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:33.553 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:33.554 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:33.554 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:33.555 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:33.555 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:33.555 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:33.556 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:33.556 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:33.557 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:18:34.299 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:34.297+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:34.416 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:34.414+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:35.251 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:35.249+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:35.401 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:35.399+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:36.246 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:36.244+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:36.442 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:36.440+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:37.288 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:37.286+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:37.474 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:37.472+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:38.326 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:38.324+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:38.506 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:38.504+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:39.318 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:39.315+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:39.482 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:39.480+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:40.350 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:40.348+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:40.434 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:40.431+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:41.335 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:41.332+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:41.394 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:41.391+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:42.358 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:42.355+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:42.391 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:42.388+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:43.326 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:43.323+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:43.436 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:43.433+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:44.308 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:44.305+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:44.476 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:44.473+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:45.329 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:45.326+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:45.482 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:45.479+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:46.305 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:46.302+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:46.494 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:46.491+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:47.342 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:47.339+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:47.464 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:47.461+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:48.305 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:48.302+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:48.415 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:48.413+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:49.309 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:49.306+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:49.368 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:49.365+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:50.328 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:50.325+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:50.358 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:50.355+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:51.279 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:51.276+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:51.366 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:51.363+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:52.304 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:52.301+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:52.370 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:52.367+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:53.276 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:53.273+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:53.407 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:53.405+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:53.558 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:18:53.897 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:18:53.899 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:18:53.911 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":6140,"stamp":"2023-10-18T19:18:52.639384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001496"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740768,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.20300000000000001,"5min":0.216,"15min":0.221},"min":{"1min":0.14599999999999999,"5min":0.14599999999999999,"15min":0.14199999999999999},"max":{"1min":0.254,"5min":0.38200000000000001,"15min":0.47599999999999998},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.247,"5min":0.26100000000000001,"15min":0.26200000000000001},"min":{"1min":0.19600000000000001,"5min":0.17999999999999999,"15min":0.16600000000000001},"max":{"1min":0.30499999999999999,"5min":0.45500000000000002,"15min":0.47499999999999998},"last":0.252}]},{"osd":1,"last update":"Wed Oct 18 19:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.19700000000000001,"5min":0.22600000000000001,"15min":0.23300000000000001},"min":{"1min":0.157,"5min":0.157,"15min":0.157},"max":{"1min":0.221,"5min":0.41699999999999998,"15min":0.48599999999999999},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.22,"5min":0.24099999999999999,"15min":0.24299999999999999},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.17199999999999999},"max":{"1min":0.27200000000000002,"5min":0.442,"15min":0.48699999999999999},"last":0.20999999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740769,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.21099999999999999,"5min":0.214,"15min":0.21099999999999999},"min":{"1min":0.158,"5min":0.13700000000000001,"15min":0.13200000000000001},"max":{"1min":0.39100000000000001,"5min":0.42099999999999999,"15min":0.47799999999999998},"last":0.217},{"interface":"front","average":{"1min":0.23100000000000001,"5min":0.23000000000000001,"15min":0.23100000000000001},"min":{"1min":0.18099999999999999,"5min":0.16,"15min":0.156},"max":{"1min":0.40200000000000002,"5min":0.47299999999999998,"15min":0.48399999999999999},"last":0.20000000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.23200000000000001,"15min":0.23300000000000001},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.152},"max":{"1min":0.41399999999999998,"5min":0.438,"15min":0.44600000000000001},"last":0.191},{"interface":"front","average":{"1min":0.26000000000000001,"5min":0.25900000000000001,"15min":0.251},"min":{"1min":0.21299999999999999,"5min":0.19700000000000001,"15min":0.16600000000000001},"max":{"1min":0.432,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.23799999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740769,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.224,"5min":0.219,"15min":0.219},"min":{"1min":0.153,"5min":0.153,"15min":0.152},"max":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.46200000000000002},"last":0.17799999999999999},{"interface":"front","average":{"1min":0.26100000000000001,"5min":0.252,"15min":0.251},"min":{"1min":0.191,"5min":0.17999999999999999,"15min":0.17599999999999999},"max":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.51000000000000001},"last":0.20699999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.22500000000000001,"15min":0.23400000000000001},"min":{"1min":0.17799999999999999,"5min":0.154,"15min":0.154},"max":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.44700000000000001},"last":0.185},{"interface":"front","average":{"1min":0.27000000000000002,"5min":0.26400000000000001,"15min":0.26700000000000002},"min":{"1min":0.218,"5min":0.20000000000000001,"15min":0.184},"max":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.22900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:18:53.912 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:53.913 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:53.913 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:53.914 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:53.914 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:53.914 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:53.915 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:53.915 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:53.916 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:53.916 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:18:53.916 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:18:54.276 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:54.273+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:54.392 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:54.389+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:55.247 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:55.244+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:55.359 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:55.356+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:56.198 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:56.195+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:56.339 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:56.336+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:57.183 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:57.180+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:57.330 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:57.327+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:58.164 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:58.161+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:58.350 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:58.347+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:59.195 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:18:59.192+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:18:59.392 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:18:59.389+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:00.217 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:00.214+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:00.405 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:00.402+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:01.193 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:01.190+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:01.363 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:01.360+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:02.243 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:02.240+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:02.353 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:02.350+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:03.265 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:03.262+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:03.351 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:03.348+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:04.273 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:04.270+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:04.382 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:04.379+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:05.299 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:05.296+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:05.428 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:05.425+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:06.310 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:06.307+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:06.425 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:06.423+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:07.262 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:07.259+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:07.463 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:07.460+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:08.268 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:08.265+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:08.444 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:08.441+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:09.292 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:09.289+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:09.408 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:09.405+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:10.263 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:10.260+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:10.363 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:10.359+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:11.218 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:11.215+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:11.325 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:11.322+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:12.190 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:12.188+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:12.300 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:12.298+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:13.158 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:13.156+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:13.292 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:13.290+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:13.918 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:19:14.192 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:14.190+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:14.213 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:19:14.215 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:19:14.226 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":6150,"stamp":"2023-10-18T19:19:12.641999+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001530"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740772,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.217,"5min":0.215,"15min":0.221},"min":{"1min":0.13900000000000001,"5min":0.13900000000000001,"15min":0.13900000000000001},"max":{"1min":0.29899999999999999,"5min":0.38200000000000001,"15min":0.47599999999999998},"last":0.18099999999999999},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.26100000000000001,"15min":0.26200000000000001},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.16600000000000001},"max":{"1min":0.33800000000000002,"5min":0.45500000000000002,"15min":0.47499999999999998},"last":0.254}]},{"osd":1,"last update":"Wed Oct 18 19:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.22600000000000001,"15min":0.23400000000000001},"min":{"1min":0.17799999999999999,"5min":0.157,"15min":0.157},"max":{"1min":0.31900000000000001,"5min":0.41699999999999998,"15min":0.48599999999999999},"last":0.21199999999999999},{"interface":"front","average":{"1min":0.23699999999999999,"5min":0.23999999999999999,"15min":0.24399999999999999},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.32900000000000001,"5min":0.442,"15min":0.48699999999999999},"last":0.22}]}]},{"osd":0,"up_from":8,"seq":34359740773,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.21099999999999999,"5min":0.214,"15min":0.21099999999999999},"min":{"1min":0.158,"5min":0.13700000000000001,"15min":0.13200000000000001},"max":{"1min":0.39100000000000001,"5min":0.42099999999999999,"15min":0.47799999999999998},"last":0.20599999999999999},{"interface":"front","average":{"1min":0.23100000000000001,"5min":0.23000000000000001,"15min":0.23100000000000001},"min":{"1min":0.18099999999999999,"5min":0.16,"15min":0.156},"max":{"1min":0.40200000000000002,"5min":0.47299999999999998,"15min":0.48399999999999999},"last":0.224}]},{"osd":2,"last update":"Wed Oct 18 19:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.23200000000000001,"15min":0.23300000000000001},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.152},"max":{"1min":0.41399999999999998,"5min":0.438,"15min":0.44600000000000001},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.26000000000000001,"5min":0.25900000000000001,"15min":0.251},"min":{"1min":0.21299999999999999,"5min":0.19700000000000001,"15min":0.16600000000000001},"max":{"1min":0.432,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.26300000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740773,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.224,"5min":0.219,"15min":0.219},"min":{"1min":0.153,"5min":0.153,"15min":0.152},"max":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.46200000000000002},"last":0.20599999999999999},{"interface":"front","average":{"1min":0.26100000000000001,"5min":0.252,"15min":0.251},"min":{"1min":0.191,"5min":0.17999999999999999,"15min":0.17599999999999999},"max":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.51000000000000001},"last":0.26700000000000002}]},{"osd":2,"last update":"Wed Oct 18 19:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.22500000000000001,"15min":0.23400000000000001},"min":{"1min":0.17799999999999999,"5min":0.154,"15min":0.154},"max":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.44700000000000001},"last":0.189},{"interface":"front","average":{"1min":0.27000000000000002,"5min":0.26400000000000001,"15min":0.26700000000000002},"min":{"1min":0.218,"5min":0.20000000000000001,"15min":0.184},"max":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.23899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:19:14.228 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:14.228 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:14.229 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:14.229 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:14.230 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:14.230 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:14.231 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:14.231 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:14.231 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:14.232 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:14.232 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:19:14.372 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:14.337+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:15.182 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:15.180+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:15.382 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:15.380+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:16.162 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:16.160+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:16.381 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:16.379+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:17.172 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:17.170+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:17.348 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:17.346+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:18.215 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:18.213+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:18.371 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:18.369+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:19.173 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:19.171+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:19.397 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:19.395+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:20.160 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:20.158+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:20.432 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:20.430+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:21.154 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:21.152+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:21.414 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:21.412+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:22.203 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:22.201+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:22.444 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:22.442+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:23.243 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:23.241+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:23.475 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:23.473+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:24.268 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:24.266+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:24.488 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:24.486+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:25.278 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:25.276+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:25.522 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:25.520+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:26.288 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:26.286+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:26.501 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:26.499+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:27.334 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:27.332+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:27.464 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:27.462+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:28.375 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:28.373+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:28.457 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:28.456+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:29.400 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:29.398+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:29.417 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:29.416+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:30.352 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:30.349+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:30.427 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:30.425+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:31.368 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:31.365+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:31.395 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:31.393+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:32.399 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:32.396+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:32.413 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:32.410+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:33.399 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:33.396+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:33.405 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:33.403+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:34.234 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:19:34.384 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:34.382+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:34.436 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:34.433+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:34.543 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:19:34.545 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:19:34.556 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":6160,"stamp":"2023-10-18T19:19:32.645165+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001881"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740776,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.217,"5min":0.215,"15min":0.221},"min":{"1min":0.13900000000000001,"5min":0.13900000000000001,"15min":0.13900000000000001},"max":{"1min":0.29899999999999999,"5min":0.38200000000000001,"15min":0.47599999999999998},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.26100000000000001,"15min":0.26200000000000001},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.16600000000000001},"max":{"1min":0.33800000000000002,"5min":0.45500000000000002,"15min":0.47499999999999998},"last":0.249}]},{"osd":1,"last update":"Wed Oct 18 19:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.22600000000000001,"15min":0.23400000000000001},"min":{"1min":0.17799999999999999,"5min":0.157,"15min":0.157},"max":{"1min":0.31900000000000001,"5min":0.41699999999999998,"15min":0.48599999999999999},"last":0.192},{"interface":"front","average":{"1min":0.23699999999999999,"5min":0.23999999999999999,"15min":0.24399999999999999},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.32900000000000001,"5min":0.442,"15min":0.48699999999999999},"last":0.21099999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740777,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.21099999999999999,"5min":0.214,"15min":0.21099999999999999},"min":{"1min":0.158,"5min":0.13700000000000001,"15min":0.13200000000000001},"max":{"1min":0.39100000000000001,"5min":0.42099999999999999,"15min":0.47799999999999998},"last":0.184},{"interface":"front","average":{"1min":0.23100000000000001,"5min":0.23000000000000001,"15min":0.23100000000000001},"min":{"1min":0.18099999999999999,"5min":0.16,"15min":0.156},"max":{"1min":0.40200000000000002,"5min":0.47299999999999998,"15min":0.48399999999999999},"last":0.19500000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.23200000000000001,"15min":0.23300000000000001},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.152},"max":{"1min":0.41399999999999998,"5min":0.438,"15min":0.44600000000000001},"last":0.20699999999999999},{"interface":"front","average":{"1min":0.26000000000000001,"5min":0.25900000000000001,"15min":0.251},"min":{"1min":0.21299999999999999,"5min":0.19700000000000001,"15min":0.16600000000000001},"max":{"1min":0.432,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.23400000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740777,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.224,"5min":0.219,"15min":0.219},"min":{"1min":0.153,"5min":0.153,"15min":0.152},"max":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.46200000000000002},"last":0.19},{"interface":"front","average":{"1min":0.26100000000000001,"5min":0.252,"15min":0.251},"min":{"1min":0.191,"5min":0.17999999999999999,"15min":0.17599999999999999},"max":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.51000000000000001},"last":0.219}]},{"osd":2,"last update":"Wed Oct 18 19:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.22500000000000001,"15min":0.23400000000000001},"min":{"1min":0.17799999999999999,"5min":0.154,"15min":0.154},"max":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.44700000000000001},"last":0.20799999999999999},{"interface":"front","average":{"1min":0.27000000000000002,"5min":0.26400000000000001,"15min":0.26700000000000002},"min":{"1min":0.218,"5min":0.20000000000000001,"15min":0.184},"max":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.24199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:19:34.558 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:34.558 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:34.558 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:34.559 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:34.559 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:34.560 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:34.560 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:34.560 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:34.561 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:34.561 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:34.562 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.7 2023-10-18T19:19:34.871 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.7 on osd.1 to deep-scrub 2023-10-18T19:19:34.882 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-10-18T19:19:35.192 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-10-18T19:19:35.204 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.5 2023-10-18T19:19:35.350 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:35.346+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:35.476 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:35.473+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:35.513 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.5 on osd.1 to deep-scrub 2023-10-18T19:19:35.524 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.1 2023-10-18T19:19:35.821 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.1 on osd.0 to deep-scrub 2023-10-18T19:19:35.832 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.2 2023-10-18T19:19:36.170 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.2 on osd.1 to deep-scrub 2023-10-18T19:19:36.182 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-10-18T19:19:36.328 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:36.325+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:36.454 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:36.451+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:36.521 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-10-18T19:19:36.532 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.0 2023-10-18T19:19:36.844 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2023-10-18T19:19:36.856 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.3 2023-10-18T19:19:37.154 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.3 on osd.0 to deep-scrub 2023-10-18T19:19:37.164 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.2 2023-10-18T19:19:37.345 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:37.342+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:37.462 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.2 on osd.0 to deep-scrub 2023-10-18T19:19:37.467 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:37.464+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:37.473 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-10-18T19:19:37.825 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-10-18T19:19:37.836 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:19:38.319 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:38.316+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:38.457 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:38.455+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:39.272 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:39.269+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:39.447 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:39.444+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:40.310 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:40.307+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:40.482 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:40.479+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:41.330 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:41.327+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:41.512 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:41.509+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:42.281 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:42.278+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:42.488 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:42.486+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:43.234 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:43.231+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:43.486 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:43.483+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:44.215 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:44.213+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:44.497 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:44.494+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:45.186 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:45.183+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:45.525 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:45.522+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:46.207 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:46.204+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:46.556 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:46.553+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:47.187 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:47.184+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:47.595 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:47.593+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:48.218 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:48.215+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:48.556 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:48.554+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:49.216 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:49.213+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:49.574 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:49.571+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:50.247 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:50.244+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:50.530 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:50.527+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:51.280 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:51.277+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:51.551 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:51.548+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:52.238 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:52.235+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:52.533 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:52.530+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:53.288 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:53.285+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:53.542 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:53.539+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:54.286 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:54.283+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:54.556 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:54.553+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:55.262 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:55.259+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:55.535 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:55.532+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:56.284 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:56.281+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:56.563 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:56.560+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:57.270 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:57.267+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:57.549 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:57.546+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:57.837 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:19:58.173 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:19:58.175 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:19:58.187 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":6172,"stamp":"2023-10-18T19:19:56.648618+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001791"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740780,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.217,"5min":0.215,"15min":0.221},"min":{"1min":0.13900000000000001,"5min":0.13900000000000001,"15min":0.13900000000000001},"max":{"1min":0.29899999999999999,"5min":0.38200000000000001,"15min":0.47599999999999998},"last":0.254},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.26100000000000001,"15min":0.26200000000000001},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.16600000000000001},"max":{"1min":0.33800000000000002,"5min":0.45500000000000002,"15min":0.47499999999999998},"last":0.27200000000000002}]},{"osd":1,"last update":"Wed Oct 18 19:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.22600000000000001,"15min":0.23400000000000001},"min":{"1min":0.17799999999999999,"5min":0.157,"15min":0.157},"max":{"1min":0.31900000000000001,"5min":0.41699999999999998,"15min":0.48599999999999999},"last":0.214},{"interface":"front","average":{"1min":0.23699999999999999,"5min":0.23999999999999999,"15min":0.24399999999999999},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.32900000000000001,"5min":0.442,"15min":0.48699999999999999},"last":0.23799999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740782,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.20799999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"min":{"1min":0.16,"5min":0.13700000000000001,"15min":0.13200000000000001},"max":{"1min":0.33800000000000002,"5min":0.42099999999999999,"15min":0.47799999999999998},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.223,"5min":0.22700000000000001,"15min":0.23100000000000001},"min":{"1min":0.19,"5min":0.16,"15min":0.156},"max":{"1min":0.36499999999999999,"5min":0.47299999999999998,"15min":0.48399999999999999},"last":0.31900000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.23300000000000001,"5min":0.23000000000000001,"15min":0.23400000000000001},"min":{"1min":0.19500000000000001,"5min":0.16700000000000001,"15min":0.152},"max":{"1min":0.39900000000000002,"5min":0.433,"15min":0.44600000000000001},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.255,"5min":0.25800000000000001,"15min":0.253},"min":{"1min":0.193,"5min":0.193,"15min":0.16600000000000001},"max":{"1min":0.40899999999999997,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.35899999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740783,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.20399999999999999,"5min":0.217,"15min":0.22},"min":{"1min":0.14399999999999999,"5min":0.14399999999999999,"15min":0.14399999999999999},"max":{"1min":0.28100000000000003,"5min":0.41799999999999998,"15min":0.46200000000000002},"last":0.192},{"interface":"front","average":{"1min":0.23200000000000001,"5min":0.25,"15min":0.251},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17599999999999999},"max":{"1min":0.26700000000000002,"5min":0.45500000000000002,"15min":0.51000000000000001},"last":0.23899999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.216,"5min":0.224,"15min":0.23400000000000001},"min":{"1min":0.185,"5min":0.154,"15min":0.154},"max":{"1min":0.29399999999999998,"5min":0.42799999999999999,"15min":0.44700000000000001},"last":0.17899999999999999},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.26600000000000001,"15min":0.26800000000000002},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19600000000000001},"max":{"1min":0.34399999999999997,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.217}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:19:58.191 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:58.191 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:58.192 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:58.192 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:58.192 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:58.193 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:58.193 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:58.194 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:58.194 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:58.194 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:19:58.195 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:19:58.223 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:58.220+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:58.540 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:58.537+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:59.270 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:19:59.267+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:19:59.544 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:19:59.541+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:00.257 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:00.254+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:00.549 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:00.547+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:01.270 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:01.267+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:01.503 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:01.500+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:02.263 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:02.260+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:02.461 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:02.459+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:03.310 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:03.308+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:03.484 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:03.482+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:04.353 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:04.351+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:04.483 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:04.481+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:05.399 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:05.397+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:05.451 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:05.449+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:06.395 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:06.393+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:06.423 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:06.421+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:07.394 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:07.392+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:07.460 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:07.458+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:08.420 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:08.418+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:08.427 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:08.425+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:09.390 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:09.389+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:09.418 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:09.417+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:10.387 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:10.385+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:10.411 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:10.409+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:11.366 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:11.364+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:11.406 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:11.404+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:12.406 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:12.403+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:12.410 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:12.409+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:13.379 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:13.377+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:13.393 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:13.392+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:14.357 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:14.355+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:14.429 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:14.427+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:15.340 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:15.339+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:15.437 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:15.435+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:16.324 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:16.322+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:16.478 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:16.476+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:17.333 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:17.331+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:17.479 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:17.477+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:18.197 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:20:18.339 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:18.337+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:18.438 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:18.436+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:18.508 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:20:18.510 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:20:18.523 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":6182,"stamp":"2023-10-18T19:20:16.651557+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001739"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740784,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.24199999999999999,"5min":0.218,"15min":0.222},"min":{"1min":0.17100000000000001,"5min":0.13900000000000001,"15min":0.13900000000000001},"max":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.47599999999999998},"last":0.20599999999999999},{"interface":"front","average":{"1min":0.28199999999999997,"5min":0.26300000000000001,"15min":0.26400000000000001},"min":{"1min":0.23499999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.47499999999999998},"last":0.27600000000000002}]},{"osd":1,"last update":"Wed Oct 18 19:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.251,"5min":0.22800000000000001,"15min":0.23400000000000001},"min":{"1min":0.192,"5min":0.157,"15min":0.157},"max":{"1min":0.439,"5min":0.439,"15min":0.48599999999999999},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.25900000000000001,"5min":0.23999999999999999,"15min":0.245},"min":{"1min":0.20999999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.48699999999999999},"last":0.26200000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740786,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.20799999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"min":{"1min":0.16,"5min":0.13700000000000001,"15min":0.13200000000000001},"max":{"1min":0.33800000000000002,"5min":0.42099999999999999,"15min":0.47799999999999998},"last":0.191},{"interface":"front","average":{"1min":0.223,"5min":0.22700000000000001,"15min":0.23100000000000001},"min":{"1min":0.19,"5min":0.16,"15min":0.156},"max":{"1min":0.36499999999999999,"5min":0.47299999999999998,"15min":0.48399999999999999},"last":0.26000000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.23300000000000001,"5min":0.23000000000000001,"15min":0.23400000000000001},"min":{"1min":0.19500000000000001,"5min":0.16700000000000001,"15min":0.152},"max":{"1min":0.39900000000000002,"5min":0.433,"15min":0.44600000000000001},"last":0.20300000000000001},{"interface":"front","average":{"1min":0.255,"5min":0.25800000000000001,"15min":0.253},"min":{"1min":0.193,"5min":0.193,"15min":0.16600000000000001},"max":{"1min":0.40899999999999997,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.22900000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740787,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.20399999999999999,"5min":0.217,"15min":0.22},"min":{"1min":0.14399999999999999,"5min":0.14399999999999999,"15min":0.14399999999999999},"max":{"1min":0.28100000000000003,"5min":0.41799999999999998,"15min":0.46200000000000002},"last":0.14599999999999999},{"interface":"front","average":{"1min":0.23200000000000001,"5min":0.25,"15min":0.251},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17599999999999999},"max":{"1min":0.26700000000000002,"5min":0.45500000000000002,"15min":0.51000000000000001},"last":0.17599999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.216,"5min":0.224,"15min":0.23400000000000001},"min":{"1min":0.185,"5min":0.154,"15min":0.154},"max":{"1min":0.29399999999999998,"5min":0.42799999999999999,"15min":0.44700000000000001},"last":0.186},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.26600000000000001,"15min":0.26800000000000002},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19600000000000001},"max":{"1min":0.34399999999999997,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.20499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:20:18.524 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:18.525 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:18.525 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:18.525 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:18.526 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:18.526 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:18.527 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:18.527 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:18.527 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:18.528 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:18.528 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:20:19.346 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:19.344+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:19.434 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:19.433+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:20.300 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:20.297+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:20.419 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:20.416+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:21.274 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:21.272+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:21.440 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:21.437+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:22.290 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:22.287+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:22.457 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:22.454+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:23.243 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:23.240+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:23.494 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:23.491+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:24.269 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:24.266+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:24.472 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:24.469+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:25.257 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:25.254+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:25.499 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:25.496+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:26.257 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:26.255+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:26.516 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:26.513+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:27.225 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:27.222+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:27.477 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:27.474+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:28.206 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:28.204+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:28.519 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:28.516+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:29.157 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:29.154+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:29.542 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:29.540+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:30.187 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:30.184+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:30.534 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:30.532+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:31.183 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:31.180+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:31.512 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:31.509+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:32.183 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:32.180+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:32.554 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:32.551+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:33.173 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:33.171+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:33.582 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:33.579+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:34.150 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:34.148+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:34.543 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:34.540+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:35.179 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:35.176+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:35.579 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:35.576+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:36.205 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:36.202+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:36.562 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:36.559+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:37.211 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:37.208+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:37.607 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:37.604+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:38.219 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:38.216+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:38.529 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:20:38.599 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:38.596+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:38.830 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:20:38.832 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:20:38.845 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":6193,"stamp":"2023-10-18T19:20:38.655294+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002082"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740789,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.24199999999999999,"5min":0.218,"15min":0.222},"min":{"1min":0.17100000000000001,"5min":0.13900000000000001,"15min":0.13900000000000001},"max":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.47599999999999998},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.28199999999999997,"5min":0.26300000000000001,"15min":0.26400000000000001},"min":{"1min":0.23499999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.47499999999999998},"last":0.29299999999999998}]},{"osd":1,"last update":"Wed Oct 18 19:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.251,"5min":0.22800000000000001,"15min":0.23400000000000001},"min":{"1min":0.192,"5min":0.157,"15min":0.157},"max":{"1min":0.439,"5min":0.439,"15min":0.48599999999999999},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.25900000000000001,"5min":0.23999999999999999,"15min":0.245},"min":{"1min":0.20999999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.48699999999999999},"last":0.23000000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740790,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.20799999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"min":{"1min":0.16,"5min":0.13700000000000001,"15min":0.13200000000000001},"max":{"1min":0.33800000000000002,"5min":0.42099999999999999,"15min":0.47799999999999998},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.223,"5min":0.22700000000000001,"15min":0.23100000000000001},"min":{"1min":0.19,"5min":0.16,"15min":0.156},"max":{"1min":0.36499999999999999,"5min":0.47299999999999998,"15min":0.48399999999999999},"last":0.17999999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.23300000000000001,"5min":0.23000000000000001,"15min":0.23400000000000001},"min":{"1min":0.19500000000000001,"5min":0.16700000000000001,"15min":0.152},"max":{"1min":0.39900000000000002,"5min":0.433,"15min":0.44600000000000001},"last":0.20100000000000001},{"interface":"front","average":{"1min":0.255,"5min":0.25800000000000001,"15min":0.253},"min":{"1min":0.193,"5min":0.193,"15min":0.16600000000000001},"max":{"1min":0.40899999999999997,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.223}]}]},{"osd":1,"up_from":8,"seq":34359740791,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.20399999999999999,"5min":0.217,"15min":0.22},"min":{"1min":0.14399999999999999,"5min":0.14399999999999999,"15min":0.14399999999999999},"max":{"1min":0.28100000000000003,"5min":0.41799999999999998,"15min":0.46200000000000002},"last":0.20300000000000001},{"interface":"front","average":{"1min":0.23200000000000001,"5min":0.25,"15min":0.251},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17599999999999999},"max":{"1min":0.26700000000000002,"5min":0.45500000000000002,"15min":0.51000000000000001},"last":0.23799999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.216,"5min":0.224,"15min":0.23400000000000001},"min":{"1min":0.185,"5min":0.154,"15min":0.154},"max":{"1min":0.29399999999999998,"5min":0.42799999999999999,"15min":0.44700000000000001},"last":0.21299999999999999},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.26600000000000001,"15min":0.26800000000000002},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19600000000000001},"max":{"1min":0.34399999999999997,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.26800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:20:38.846 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:38.847 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:38.847 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:38.848 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:38.848 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:38.848 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:38.849 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:38.849 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:38.849 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:38.850 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:38.850 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:20:39.265 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:39.262+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:39.613 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:39.611+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:40.221 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:40.219+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:40.580 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:40.577+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:41.262 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:41.259+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:41.617 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:41.614+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:42.255 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:42.252+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:42.609 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:42.606+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:43.302 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:43.299+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:43.589 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:43.586+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:44.309 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:44.306+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:44.548 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:44.545+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:45.264 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:45.261+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:45.540 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:45.537+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:46.313 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:46.310+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:46.500 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:46.497+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:47.352 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:47.349+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:47.456 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:47.453+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:48.348 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:48.345+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:48.448 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:48.445+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:49.331 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:49.328+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:49.438 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:49.436+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:50.332 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:50.329+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:50.458 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:50.456+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:51.325 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:51.322+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:51.457 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:51.455+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:52.370 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:52.368+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:52.434 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:52.431+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:53.325 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:53.323+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:53.410 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:53.408+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:54.294 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:54.293+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:54.365 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:54.363+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:55.342 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:55.340+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:55.414 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:55.412+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:56.309 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:56.307+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:56.442 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:56.437+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:57.317 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:57.315+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:57.463 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:57.461+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:58.350 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:58.348+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:58.500 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:58.498+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:58.852 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:20:59.144 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:20:59.146 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:20:59.157 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":6203,"stamp":"2023-10-18T19:20:58.658830+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002076"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740793,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.24199999999999999,"5min":0.218,"15min":0.222},"min":{"1min":0.17100000000000001,"5min":0.13900000000000001,"15min":0.13900000000000001},"max":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.47599999999999998},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.28199999999999997,"5min":0.26300000000000001,"15min":0.26400000000000001},"min":{"1min":0.23499999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.47499999999999998},"last":0.28999999999999998}]},{"osd":1,"last update":"Wed Oct 18 19:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.251,"5min":0.22800000000000001,"15min":0.23400000000000001},"min":{"1min":0.192,"5min":0.157,"15min":0.157},"max":{"1min":0.439,"5min":0.439,"15min":0.48599999999999999},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.25900000000000001,"5min":0.23999999999999999,"15min":0.245},"min":{"1min":0.20999999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.48699999999999999},"last":0.26800000000000002}]}]},{"osd":0,"up_from":8,"seq":34359740794,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.21099999999999999,"15min":0.214},"min":{"1min":0.16,"5min":0.13700000000000001,"15min":0.13700000000000001},"max":{"1min":0.33700000000000002,"5min":0.42099999999999999,"15min":0.47799999999999998},"last":0.19600000000000001},{"interface":"front","average":{"1min":0.23100000000000001,"5min":0.22600000000000001,"15min":0.23100000000000001},"min":{"1min":0.17799999999999999,"5min":0.16600000000000001,"15min":0.156},"max":{"1min":0.34499999999999997,"5min":0.40200000000000002,"15min":0.48399999999999999},"last":0.22800000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.24199999999999999,"5min":0.22900000000000001,"15min":0.23499999999999999},"min":{"1min":0.187,"5min":0.16700000000000001,"15min":0.152},"max":{"1min":0.39400000000000002,"5min":0.41399999999999998,"15min":0.44600000000000001},"last":0.217},{"interface":"front","average":{"1min":0.26900000000000002,"5min":0.25700000000000001,"15min":0.255},"min":{"1min":0.20899999999999999,"5min":0.193,"15min":0.16600000000000001},"max":{"1min":0.39200000000000002,"5min":0.432,"15min":0.54600000000000004},"last":0.251}]}]},{"osd":1,"up_from":8,"seq":34359740795,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.20899999999999999,"5min":0.214,"15min":0.219},"min":{"1min":0.14599999999999999,"5min":0.14399999999999999,"15min":0.14399999999999999},"max":{"1min":0.29599999999999999,"5min":0.41799999999999998,"15min":0.46200000000000002},"last":0.249},{"interface":"front","average":{"1min":0.23599999999999999,"5min":0.246,"15min":0.251},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.32800000000000001,"5min":0.45500000000000002,"15min":0.51000000000000001},"last":0.27900000000000003}]},{"osd":2,"last update":"Wed Oct 18 19:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.21299999999999999,"5min":0.219,"15min":0.23200000000000001},"min":{"1min":0.16700000000000001,"5min":0.154,"15min":0.154},"max":{"1min":0.28299999999999997,"5min":0.42799999999999999,"15min":0.44700000000000001},"last":0.189},{"interface":"front","average":{"1min":0.249,"5min":0.26000000000000001,"15min":0.26700000000000002},"min":{"1min":0.20499999999999999,"5min":0.19900000000000001,"15min":0.19600000000000001},"max":{"1min":0.33400000000000002,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.308}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:20:59.159 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:59.159 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:59.159 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:59.160 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:59.160 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:59.161 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:59.161 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:59.161 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:59.162 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:59.162 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:20:59.163 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:20:59.310 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:20:59.308+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:20:59.518 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:20:59.517+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:00.350 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:00.349+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:00.507 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:00.505+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:01.330 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:01.328+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:01.502 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:01.500+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:02.360 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:02.359+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:02.522 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:02.521+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:03.345 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:03.343+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:03.566 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:03.564+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:04.316 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:04.314+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:04.568 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:04.567+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:05.351 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:05.349+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:05.593 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:05.591+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:06.371 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:06.369+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:06.635 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:06.633+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:07.354 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:07.353+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:07.667 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:07.665+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:08.359 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:08.357+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:08.657 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:08.655+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:09.363 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:09.361+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:09.633 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:09.631+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:10.315 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:10.312+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:10.618 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:10.615+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:11.363 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:11.360+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:11.641 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:11.638+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:12.403 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:12.400+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:12.620 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:12.617+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:13.374 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:13.371+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:13.608 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:13.605+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:14.412 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:14.410+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:14.655 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:14.652+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:15.375 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:15.373+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:15.652 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:15.648+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:16.343 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:16.341+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:16.622 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:16.619+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:17.346 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:17.343+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:17.646 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:17.643+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:18.377 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:18.374+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:18.694 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:18.691+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:19.164 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:21:19.394 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:19.392+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:19.460 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:21:19.462 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:21:19.473 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":6213,"stamp":"2023-10-18T19:21:18.662167+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002029"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740797,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:20:56 2023","interfaces":[{"interface":"back","average":{"1min":0.23499999999999999,"5min":0.224,"15min":0.224},"min":{"1min":0.16600000000000001,"5min":0.13900000000000001,"15min":0.13900000000000001},"max":{"1min":0.438,"5min":0.438,"15min":0.47599999999999998},"last":0.17999999999999999},{"interface":"front","average":{"1min":0.28599999999999998,"5min":0.26800000000000002,"15min":0.26500000000000001},"min":{"1min":0.23100000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.23100000000000001}]},{"osd":1,"last update":"Wed Oct 18 19:20:56 2023","interfaces":[{"interface":"back","average":{"1min":0.25,"5min":0.23100000000000001,"15min":0.23499999999999999},"min":{"1min":0.189,"5min":0.157,"15min":0.157},"max":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.48599999999999999},"last":0.215},{"interface":"front","average":{"1min":0.26600000000000001,"5min":0.246,"15min":0.247},"min":{"1min":0.20399999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.20699999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740798,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.21099999999999999,"15min":0.214},"min":{"1min":0.16,"5min":0.13700000000000001,"15min":0.13700000000000001},"max":{"1min":0.33700000000000002,"5min":0.42099999999999999,"15min":0.47799999999999998},"last":0.19400000000000001},{"interface":"front","average":{"1min":0.23100000000000001,"5min":0.22600000000000001,"15min":0.23100000000000001},"min":{"1min":0.17799999999999999,"5min":0.16600000000000001,"15min":0.156},"max":{"1min":0.34499999999999997,"5min":0.40200000000000002,"15min":0.48399999999999999},"last":0.21199999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.24199999999999999,"5min":0.22900000000000001,"15min":0.23499999999999999},"min":{"1min":0.187,"5min":0.16700000000000001,"15min":0.152},"max":{"1min":0.39400000000000002,"5min":0.41399999999999998,"15min":0.44600000000000001},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.26900000000000002,"5min":0.25700000000000001,"15min":0.255},"min":{"1min":0.20899999999999999,"5min":0.193,"15min":0.16600000000000001},"max":{"1min":0.39200000000000002,"5min":0.432,"15min":0.54600000000000004},"last":0.26200000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740799,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.20899999999999999,"5min":0.214,"15min":0.219},"min":{"1min":0.14599999999999999,"5min":0.14399999999999999,"15min":0.14399999999999999},"max":{"1min":0.29599999999999999,"5min":0.41799999999999998,"15min":0.46200000000000002},"last":0.19900000000000001},{"interface":"front","average":{"1min":0.23599999999999999,"5min":0.246,"15min":0.251},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.32800000000000001,"5min":0.45500000000000002,"15min":0.51000000000000001},"last":0.23599999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.21299999999999999,"5min":0.219,"15min":0.23200000000000001},"min":{"1min":0.16700000000000001,"5min":0.154,"15min":0.154},"max":{"1min":0.28299999999999997,"5min":0.42799999999999999,"15min":0.44700000000000001},"last":0.187},{"interface":"front","average":{"1min":0.249,"5min":0.26000000000000001,"15min":0.26700000000000002},"min":{"1min":0.20499999999999999,"5min":0.19900000000000001,"15min":0.19600000000000001},"max":{"1min":0.33400000000000002,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.25900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:21:19.475 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:21:19.475 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:21:19.476 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:21:19.476 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:21:19.477 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:21:19.477 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:21:19.478 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:21:19.478 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:21:19.478 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:21:19.479 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:21:19.479 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:21:19.674 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:19.671+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:20.420 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:20.417+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:20.714 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:20.712+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:21.398 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:21.395+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:21.714 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:21.712+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:22.354 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:22.352+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:22.693 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:22.690+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:23.403 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:23.400+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:23.691 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:23.688+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:24.409 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:24.406+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:24.725 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:24.722+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:25.425 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:25.422+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:25.697 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:25.694+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:26.434 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:26.431+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:26.734 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:26.731+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:27.467 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:27.464+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:27.694 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:27.690+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:28.503 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:28.500+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:28.685 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:28.682+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:29.504 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:29.501+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:29.647 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:29.644+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:30.461 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:30.458+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:30.689 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:30.686+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:31.459 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:31.456+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:31.724 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:31.721+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:32.423 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:32.420+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:32.753 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:32.750+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:33.441 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:33.439+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:33.780 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:33.777+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:34.420 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:34.417+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:34.739 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:34.736+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:35.404 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:35.401+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:35.694 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:35.691+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:36.445 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:36.442+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:36.663 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:36.660+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:37.408 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:37.406+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:37.681 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:37.678+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:38.413 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:38.411+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:38.636 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:38.633+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:39.405 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:39.402+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:39.480 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:21:39.678 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:39.675+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:39.803 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:21:39.804 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:21:39.816 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":6223,"stamp":"2023-10-18T19:21:38.665117+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001717"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740801,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:20:56 2023","interfaces":[{"interface":"back","average":{"1min":0.23499999999999999,"5min":0.224,"15min":0.224},"min":{"1min":0.16600000000000001,"5min":0.13900000000000001,"15min":0.13900000000000001},"max":{"1min":0.438,"5min":0.438,"15min":0.47599999999999998},"last":0.192},{"interface":"front","average":{"1min":0.28599999999999998,"5min":0.26800000000000002,"15min":0.26500000000000001},"min":{"1min":0.23100000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.27100000000000002}]},{"osd":1,"last update":"Wed Oct 18 19:20:56 2023","interfaces":[{"interface":"back","average":{"1min":0.25,"5min":0.23100000000000001,"15min":0.23499999999999999},"min":{"1min":0.189,"5min":0.157,"15min":0.157},"max":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.48599999999999999},"last":0.223},{"interface":"front","average":{"1min":0.26600000000000001,"5min":0.246,"15min":0.247},"min":{"1min":0.20399999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.25600000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740802,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.21099999999999999,"15min":0.214},"min":{"1min":0.16,"5min":0.13700000000000001,"15min":0.13700000000000001},"max":{"1min":0.33700000000000002,"5min":0.42099999999999999,"15min":0.47799999999999998},"last":0.20300000000000001},{"interface":"front","average":{"1min":0.23100000000000001,"5min":0.22600000000000001,"15min":0.23100000000000001},"min":{"1min":0.17799999999999999,"5min":0.16600000000000001,"15min":0.156},"max":{"1min":0.34499999999999997,"5min":0.40200000000000002,"15min":0.48399999999999999},"last":0.247}]},{"osd":2,"last update":"Wed Oct 18 19:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.24199999999999999,"5min":0.22900000000000001,"15min":0.23499999999999999},"min":{"1min":0.187,"5min":0.16700000000000001,"15min":0.152},"max":{"1min":0.39400000000000002,"5min":0.41399999999999998,"15min":0.44600000000000001},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.26900000000000002,"5min":0.25700000000000001,"15min":0.255},"min":{"1min":0.20899999999999999,"5min":0.193,"15min":0.16600000000000001},"max":{"1min":0.39200000000000002,"5min":0.432,"15min":0.54600000000000004},"last":0.28699999999999998}]}]},{"osd":1,"up_from":8,"seq":34359740803,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.20899999999999999,"5min":0.214,"15min":0.219},"min":{"1min":0.14599999999999999,"5min":0.14399999999999999,"15min":0.14399999999999999},"max":{"1min":0.29599999999999999,"5min":0.41799999999999998,"15min":0.46200000000000002},"last":0.17399999999999999},{"interface":"front","average":{"1min":0.23599999999999999,"5min":0.246,"15min":0.251},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.32800000000000001,"5min":0.45500000000000002,"15min":0.51000000000000001},"last":0.20300000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.21299999999999999,"5min":0.219,"15min":0.23200000000000001},"min":{"1min":0.16700000000000001,"5min":0.154,"15min":0.154},"max":{"1min":0.28299999999999997,"5min":0.42799999999999999,"15min":0.44700000000000001},"last":0.18099999999999999},{"interface":"front","average":{"1min":0.249,"5min":0.26000000000000001,"15min":0.26700000000000002},"min":{"1min":0.20499999999999999,"5min":0.19900000000000001,"15min":0.19600000000000001},"max":{"1min":0.33400000000000002,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.23499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:21:39.817 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:21:39.818 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:21:39.818 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:21:39.818 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:21:39.819 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:21:39.819 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:21:39.820 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:21:39.820 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:21:39.820 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:21:39.821 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:21:39.821 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.7 2023-10-18T19:21:40.135 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.7 on osd.1 to deep-scrub 2023-10-18T19:21:40.149 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-10-18T19:21:40.452 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:40.449+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:40.493 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-10-18T19:21:40.505 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.5 2023-10-18T19:21:40.694 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:40.692+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:40.805 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.5 on osd.1 to deep-scrub 2023-10-18T19:21:40.817 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.1 2023-10-18T19:21:41.168 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.1 on osd.0 to deep-scrub 2023-10-18T19:21:41.179 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.2 2023-10-18T19:21:41.475 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:41.472+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:41.486 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.2 on osd.1 to deep-scrub 2023-10-18T19:21:41.498 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-10-18T19:21:41.733 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:41.730+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:41.848 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-10-18T19:21:41.860 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.0 2023-10-18T19:21:42.155 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2023-10-18T19:21:42.166 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.3 2023-10-18T19:21:42.460 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 3.3 on osd.0 to deep-scrub 2023-10-18T19:21:42.471 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.2 2023-10-18T19:21:42.501 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:42.498+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:42.774 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:42.772+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:42.812 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.2 on osd.0 to deep-scrub 2023-10-18T19:21:42.825 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-10-18T19:21:43.174 INFO:teuthology.orchestra.run.smithi082.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-10-18T19:21:43.185 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:21:43.473 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:43.471+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:43.762 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:43.760+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:44.435 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:44.433+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:44.723 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:44.721+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:45.480 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:45.478+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:45.729 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:45.727+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:46.514 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:46.512+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:46.695 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:46.693+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:47.538 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:47.536+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:47.731 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:47.729+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:48.504 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:48.502+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:48.703 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:48.702+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:49.496 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:49.494+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:49.722 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:49.720+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:50.480 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:50.478+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:50.704 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:50.702+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:51.444 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:51.442+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:51.666 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:51.664+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:52.451 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:52.449+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:52.701 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:52.700+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:53.444 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:53.442+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:53.752 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:53.750+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:54.458 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:54.456+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:54.712 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:54.710+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:55.441 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:55.440+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:55.738 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:55.736+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:56.447 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:56.445+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:56.758 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:56.756+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:57.492 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:57.490+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:57.767 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:57.765+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:58.515 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:58.513+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:58.731 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:58.729+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:59.550 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:21:59.545+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:21:59.707 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:21:59.705+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:00.516 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:00.514+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:00.670 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:00.668+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:01.533 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:01.530+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:01.710 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:01.707+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:02.516 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:02.513+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:02.707 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:02.704+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:03.187 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:22:03.468 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:03.465+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:03.525 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:22:03.527 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:22:03.537 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":6235,"stamp":"2023-10-18T19:22:02.668493+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001657"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740806,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:21:57 2023","interfaces":[{"interface":"back","average":{"1min":0.25,"5min":0.22900000000000001,"15min":0.22500000000000001},"min":{"1min":0.17299999999999999,"5min":0.13900000000000001,"15min":0.13900000000000001},"max":{"1min":0.41499999999999998,"5min":0.438,"15min":0.47599999999999998},"last":0.17599999999999999},{"interface":"front","average":{"1min":0.29199999999999998,"5min":0.27200000000000002,"15min":0.26600000000000001},"min":{"1min":0.23100000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.44700000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.25800000000000001}]},{"osd":1,"last update":"Wed Oct 18 19:21:57 2023","interfaces":[{"interface":"back","average":{"1min":0.248,"5min":0.23400000000000001,"15min":0.23599999999999999},"min":{"1min":0.19800000000000001,"5min":0.157,"15min":0.157},"max":{"1min":0.39600000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.249,"15min":0.247},"min":{"1min":0.20399999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.42699999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.23000000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740807,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.21199999999999999,"5min":0.20999999999999999,"15min":0.215},"min":{"1min":0.16900000000000001,"5min":0.13700000000000001,"15min":0.13700000000000001},"max":{"1min":0.28199999999999997,"5min":0.39100000000000001,"15min":0.47799999999999998},"last":0.20799999999999999},{"interface":"front","average":{"1min":0.23000000000000001,"5min":0.22600000000000001,"15min":0.23200000000000001},"min":{"1min":0.18099999999999999,"5min":0.16800000000000001,"15min":0.156},"max":{"1min":0.29099999999999998,"5min":0.40200000000000002,"15min":0.48399999999999999},"last":0.19500000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.22800000000000001,"15min":0.23499999999999999},"min":{"1min":0.19600000000000001,"5min":0.16700000000000001,"15min":0.152},"max":{"1min":0.30299999999999999,"5min":0.41399999999999998,"15min":0.44600000000000001},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.25800000000000001,"5min":0.25700000000000001,"15min":0.25600000000000001},"min":{"1min":0.188,"5min":0.188,"15min":0.16600000000000001},"max":{"1min":0.31900000000000001,"5min":0.432,"15min":0.54600000000000004},"last":0.23799999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740808,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.22900000000000001,"5min":0.215,"15min":0.219},"min":{"1min":0.152,"5min":0.14399999999999999,"15min":0.14399999999999999},"max":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.46200000000000002},"last":0.21299999999999999},{"interface":"front","average":{"1min":0.30499999999999999,"5min":0.25600000000000001,"15min":0.253},"min":{"1min":0.185,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.25800000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.224,"5min":0.217,"15min":0.23000000000000001},"min":{"1min":0.16700000000000001,"5min":0.154,"15min":0.154},"max":{"1min":0.40799999999999997,"5min":0.42799999999999999,"15min":0.44700000000000001},"last":0.19900000000000001},{"interface":"front","average":{"1min":0.26600000000000001,"5min":0.26100000000000001,"15min":0.26500000000000001},"min":{"1min":0.20000000000000001,"5min":0.19900000000000001,"15min":0.19600000000000001},"max":{"1min":0.39100000000000001,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.28699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:22:03.539 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:03.540 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:03.540 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:03.540 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:03.541 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:03.541 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:03.541 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:03.542 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:03.542 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:03.543 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:03.543 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:22:03.661 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:03.658+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:04.491 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:04.488+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:04.617 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:04.614+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:05.531 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:05.528+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:05.573 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:05.570+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:06.546 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:06.543+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:06.560 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:06.557+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:07.530 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:07.527+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:07.579 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:07.576+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:08.546 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:08.543+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:08.596 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:08.593+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:09.587 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:09.584+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:09.599 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:09.596+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:10.549 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:10.546+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:10.621 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:10.618+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:11.523 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:11.520+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:11.643 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:11.641+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:12.546 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:12.543+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:12.602 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:12.599+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:13.592 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:13.589+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:13.630 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:13.627+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:14.637 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:14.635+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:14.672 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:14.669+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:15.675 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:15.673+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:15.682 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:15.679+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:16.692 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:16.689+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:16.731 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:16.728+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:17.689 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:17.686+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:17.750 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:17.747+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:18.693 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:18.690+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:18.723 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:18.720+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:19.685 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:19.682+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:19.747 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:19.745+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:20.686 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:20.683+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:20.762 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:20.759+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:21.645 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:21.643+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:21.730 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:21.727+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:22.602 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:22.599+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:22.721 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:22.718+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:23.544 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:22:23.559 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:23.556+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:23.708 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:23.692+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:23.875 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:22:23.878 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:22:23.890 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":6245,"stamp":"2023-10-18T19:22:22.671530+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001705"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740810,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:21:57 2023","interfaces":[{"interface":"back","average":{"1min":0.25,"5min":0.22900000000000001,"15min":0.22500000000000001},"min":{"1min":0.17299999999999999,"5min":0.13900000000000001,"15min":0.13900000000000001},"max":{"1min":0.41499999999999998,"5min":0.438,"15min":0.47599999999999998},"last":0.182},{"interface":"front","average":{"1min":0.29199999999999998,"5min":0.27200000000000002,"15min":0.26600000000000001},"min":{"1min":0.23100000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.44700000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.24099999999999999}]},{"osd":1,"last update":"Wed Oct 18 19:21:57 2023","interfaces":[{"interface":"back","average":{"1min":0.248,"5min":0.23400000000000001,"15min":0.23599999999999999},"min":{"1min":0.19800000000000001,"5min":0.157,"15min":0.157},"max":{"1min":0.39600000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.254},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.249,"15min":0.247},"min":{"1min":0.20399999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.42699999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.22900000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740811,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.21199999999999999,"5min":0.20999999999999999,"15min":0.215},"min":{"1min":0.16900000000000001,"5min":0.13700000000000001,"15min":0.13700000000000001},"max":{"1min":0.28199999999999997,"5min":0.39100000000000001,"15min":0.47799999999999998},"last":0.22},{"interface":"front","average":{"1min":0.23000000000000001,"5min":0.22600000000000001,"15min":0.23200000000000001},"min":{"1min":0.18099999999999999,"5min":0.16800000000000001,"15min":0.156},"max":{"1min":0.29099999999999998,"5min":0.40200000000000002,"15min":0.48399999999999999},"last":0.192}]},{"osd":2,"last update":"Wed Oct 18 19:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.22800000000000001,"15min":0.23499999999999999},"min":{"1min":0.19600000000000001,"5min":0.16700000000000001,"15min":0.152},"max":{"1min":0.30299999999999999,"5min":0.41399999999999998,"15min":0.44600000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.25800000000000001,"5min":0.25700000000000001,"15min":0.25600000000000001},"min":{"1min":0.188,"5min":0.188,"15min":0.16600000000000001},"max":{"1min":0.31900000000000001,"5min":0.432,"15min":0.54600000000000004},"last":0.23499999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740812,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.22900000000000001,"5min":0.215,"15min":0.219},"min":{"1min":0.152,"5min":0.14399999999999999,"15min":0.14399999999999999},"max":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.46200000000000002},"last":0.19500000000000001},{"interface":"front","average":{"1min":0.30499999999999999,"5min":0.25600000000000001,"15min":0.253},"min":{"1min":0.185,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.223}]},{"osd":2,"last update":"Wed Oct 18 19:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.224,"5min":0.217,"15min":0.23000000000000001},"min":{"1min":0.16700000000000001,"5min":0.154,"15min":0.154},"max":{"1min":0.40799999999999997,"5min":0.42799999999999999,"15min":0.44700000000000001},"last":0.185},{"interface":"front","average":{"1min":0.26600000000000001,"5min":0.26100000000000001,"15min":0.26500000000000001},"min":{"1min":0.20000000000000001,"5min":0.19900000000000001,"15min":0.19600000000000001},"max":{"1min":0.39100000000000001,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.24299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:22:23.892 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:23.892 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:23.892 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:23.893 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:23.893 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:23.894 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:23.894 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:23.895 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:23.895 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:23.895 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:23.896 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:22:24.594 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:24.591+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:24.733 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:24.730+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:25.598 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:25.595+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:25.697 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:25.694+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:26.596 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:26.594+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:26.697 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:26.694+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:27.621 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:27.619+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:27.671 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:27.669+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:28.609 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:28.606+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:28.710 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:28.707+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:29.592 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:29.589+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:29.708 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:29.705+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:30.592 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:30.589+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:30.721 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:30.719+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:31.613 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:31.610+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:31.716 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:31.713+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:32.611 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:32.609+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:32.689 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:32.687+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:33.586 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:33.583+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:33.655 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:33.652+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:34.559 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:34.557+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:34.613 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:34.611+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:35.520 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:35.518+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:35.624 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:35.622+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:36.558 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:36.556+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:36.674 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:36.672+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:37.589 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:37.587+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:37.712 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:37.710+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:38.545 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:38.543+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:38.711 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:38.709+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:39.561 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:39.559+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:39.689 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:39.687+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:40.609 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:40.607+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:40.738 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:40.736+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:41.575 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:41.573+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:41.740 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:41.738+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:42.602 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:42.600+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:42.703 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:42.701+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:43.561 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:43.559+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:43.727 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:43.725+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:43.897 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:22:44.191 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:22:44.193 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:22:44.205 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":6255,"stamp":"2023-10-18T19:22:42.674456+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001844"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740814,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:21:57 2023","interfaces":[{"interface":"back","average":{"1min":0.25,"5min":0.22900000000000001,"15min":0.22500000000000001},"min":{"1min":0.17299999999999999,"5min":0.13900000000000001,"15min":0.13900000000000001},"max":{"1min":0.41499999999999998,"5min":0.438,"15min":0.47599999999999998},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.29199999999999998,"5min":0.27200000000000002,"15min":0.26600000000000001},"min":{"1min":0.23100000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.44700000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.23300000000000001}]},{"osd":1,"last update":"Wed Oct 18 19:21:57 2023","interfaces":[{"interface":"back","average":{"1min":0.248,"5min":0.23400000000000001,"15min":0.23599999999999999},"min":{"1min":0.19800000000000001,"5min":0.157,"15min":0.157},"max":{"1min":0.39600000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.249,"15min":0.247},"min":{"1min":0.20399999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.42699999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.24399999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740815,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.21199999999999999,"5min":0.20999999999999999,"15min":0.215},"min":{"1min":0.16900000000000001,"5min":0.13700000000000001,"15min":0.13700000000000001},"max":{"1min":0.28199999999999997,"5min":0.39100000000000001,"15min":0.47799999999999998},"last":0.25},{"interface":"front","average":{"1min":0.23000000000000001,"5min":0.22600000000000001,"15min":0.23200000000000001},"min":{"1min":0.18099999999999999,"5min":0.16800000000000001,"15min":0.156},"max":{"1min":0.29099999999999998,"5min":0.40200000000000002,"15min":0.48399999999999999},"last":0.20699999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.22800000000000001,"15min":0.23499999999999999},"min":{"1min":0.19600000000000001,"5min":0.16700000000000001,"15min":0.152},"max":{"1min":0.30299999999999999,"5min":0.41399999999999998,"15min":0.44600000000000001},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.25800000000000001,"5min":0.25700000000000001,"15min":0.25600000000000001},"min":{"1min":0.188,"5min":0.188,"15min":0.16600000000000001},"max":{"1min":0.31900000000000001,"5min":0.432,"15min":0.54600000000000004},"last":0.26300000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740816,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.22900000000000001,"5min":0.215,"15min":0.219},"min":{"1min":0.152,"5min":0.14399999999999999,"15min":0.14399999999999999},"max":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.46200000000000002},"last":0.217},{"interface":"front","average":{"1min":0.30499999999999999,"5min":0.25600000000000001,"15min":0.253},"min":{"1min":0.185,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.252}]},{"osd":2,"last update":"Wed Oct 18 19:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.224,"5min":0.217,"15min":0.23000000000000001},"min":{"1min":0.16700000000000001,"5min":0.154,"15min":0.154},"max":{"1min":0.40799999999999997,"5min":0.42799999999999999,"15min":0.44700000000000001},"last":0.20599999999999999},{"interface":"front","average":{"1min":0.26600000000000001,"5min":0.26100000000000001,"15min":0.26500000000000001},"min":{"1min":0.20000000000000001,"5min":0.19900000000000001,"15min":0.19600000000000001},"max":{"1min":0.39100000000000001,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.28199999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:22:44.206 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:44.207 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:44.207 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:44.208 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:44.208 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:44.208 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:44.209 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:44.209 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:44.210 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:44.210 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:22:44.210 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:22:44.539 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:44.537+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:44.732 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:44.730+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:45.579 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:45.577+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:45.762 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:45.760+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:46.533 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:46.532+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:46.759 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:46.757+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:47.537 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:47.535+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:47.752 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:47.750+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:48.545 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:48.543+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:48.796 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:48.794+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:49.573 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:49.571+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:49.815 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:49.812+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:50.524 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:50.523+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:50.785 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:50.783+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:51.490 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:51.489+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:51.832 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:51.829+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:52.476 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:52.473+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:52.810 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:52.808+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:53.519 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:53.516+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:53.827 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:53.824+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:54.479 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:54.476+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:54.861 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:54.858+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:55.435 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:55.432+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:55.853 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:55.850+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:56.443 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:56.440+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:56.862 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:56.859+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:57.493 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:57.490+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:57.850 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:57.848+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:58.456 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:58.453+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:58.806 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:58.803+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:59.451 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:22:59.448+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:22:59.769 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:22:59.766+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:00.463 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:23:00.460+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:00.739 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:23:00.736+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:01.444 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:23:01.441+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:01.759 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:23:01.756+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:02.464 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:23:02.461+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:02.809 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:23:02.806+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:03.497 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:23:03.494+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:03.785 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:23:03.782+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:04.212 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:23:04.496 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:23:04.493+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:04.521 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:23:04.523 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:23:04.535 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":6265,"stamp":"2023-10-18T19:23:02.677625+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001930"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740818,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:22:59 2023","interfaces":[{"interface":"back","average":{"1min":0.219,"5min":0.23200000000000001,"15min":0.224},"min":{"1min":0.155,"5min":0.13900000000000001,"15min":0.13900000000000001},"max":{"1min":0.34999999999999998,"5min":0.438,"15min":0.47599999999999998},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.26000000000000001,"5min":0.27500000000000002,"15min":0.26600000000000001},"min":{"1min":0.21299999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.39500000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.27800000000000002}]},{"osd":1,"last update":"Wed Oct 18 19:22:59 2023","interfaces":[{"interface":"back","average":{"1min":0.23499999999999999,"5min":0.24099999999999999,"15min":0.23499999999999999},"min":{"1min":0.183,"5min":0.17799999999999999,"15min":0.157},"max":{"1min":0.38,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.219},{"interface":"front","average":{"1min":0.24299999999999999,"5min":0.253,"15min":0.246},"min":{"1min":0.19400000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.35899999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.248}]}]},{"osd":0,"up_from":8,"seq":34359740819,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:22:58 2023","interfaces":[{"interface":"back","average":{"1min":0.23799999999999999,"5min":0.219,"15min":0.218},"min":{"1min":0.14899999999999999,"5min":0.14899999999999999,"15min":0.13700000000000001},"max":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.47799999999999998},"last":0.221},{"interface":"front","average":{"1min":0.246,"5min":0.23200000000000001,"15min":0.23400000000000001},"min":{"1min":0.182,"5min":0.17799999999999999,"15min":0.156},"max":{"1min":0.443,"5min":0.443,"15min":0.48399999999999999},"last":0.24199999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:22:58 2023","interfaces":[{"interface":"back","average":{"1min":0.249,"5min":0.23499999999999999,"15min":0.23699999999999999},"min":{"1min":0.17199999999999999,"5min":0.16700000000000001,"15min":0.152},"max":{"1min":0.435,"5min":0.435,"15min":0.44600000000000001},"last":0.215},{"interface":"front","average":{"1min":0.27200000000000002,"5min":0.26200000000000001,"15min":0.25800000000000001},"min":{"1min":0.193,"5min":0.188,"15min":0.16600000000000001},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.54600000000000004},"last":0.27300000000000002}]}]},{"osd":1,"up_from":8,"seq":34359740820,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:22:53 2023","interfaces":[{"interface":"back","average":{"1min":0.21199999999999999,"5min":0.215,"15min":0.22},"min":{"1min":0.14799999999999999,"5min":0.14399999999999999,"15min":0.14399999999999999},"max":{"1min":0.38800000000000001,"5min":0.42199999999999999,"15min":0.46200000000000002},"last":0.19800000000000001},{"interface":"front","average":{"1min":0.248,"5min":0.25600000000000001,"15min":0.254},"min":{"1min":0.17699999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.432,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.23100000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:22:53 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.221,"15min":0.23000000000000001},"min":{"1min":0.161,"5min":0.161,"15min":0.154},"max":{"1min":0.439,"5min":0.439,"15min":0.44700000000000001},"last":0.187},{"interface":"front","average":{"1min":0.25900000000000001,"5min":0.26100000000000001,"15min":0.26600000000000001},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.41599999999999998,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.25700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:23:04.537 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:23:04.537 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:23:04.537 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:23:04.538 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:23:04.538 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:23:04.539 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:23:04.539 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:23:04.539 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:23:04.540 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:23:04.540 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:23:04.540 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T19:23:04.745 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:23:04.742+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:05.472 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:23:05.469+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:05.733 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:23:05.731+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:06.457 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:23:06.454+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:06.745 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:23:06.742+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:07.446 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:23:07.443+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:07.729 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:23:07.724+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:08.430 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:23:08.427+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:08.679 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:23:08.676+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:09.400 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:23:09.397+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:09.728 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:23:09.725+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:10.430 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:23:10.427+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:10.732 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:23:10.729+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:11.403 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:23:11.400+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:11.752 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:23:11.749+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:12.416 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:23:12.413+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:12.753 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:23:12.750+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:13.445 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:23:13.442+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:13.714 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:23:13.712+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:14.443 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:23:14.440+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:14.700 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:23:14.697+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:15.399 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:23:15.396+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:15.668 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:23:15.666+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:16.396 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:23:16.393+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:16.716 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:23:16.713+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:17.435 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:23:17.432+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:17.679 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:23:17.676+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:18.458 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:23:18.455+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:18.699 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:23:18.696+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:19.428 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:23:19.425+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:19.716 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:23:19.713+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:20.403 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:23:20.400+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:20.734 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:23:20.731+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:21.439 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:23:21.436+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:21.688 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:23:21.685+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:22.414 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:23:22.411+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:22.661 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:23:22.658+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:23.378 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:23:23.375+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:23.678 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:23:23.675+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:24.364 INFO:tasks.ceph.osd.0.smithi082.stderr:2023-10-18T19:23:24.361+0000 7fa30c07e700 -1 osd.0 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:1509 3.3 3:f88d5893:test1::rbd_data.295244d43cbc.00000000000000b1:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2482176~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:24.541 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T19:23:24.660 INFO:tasks.ceph.osd.1.smithi082.stderr:2023-10-18T19:23:24.657+0000 7f1661185700 -1 osd.1 154 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10581.0:707 3.0 3:24fd14fa:test1::rbd_data.295244d43cbc.0000000000000082:head [write 921600~4096 [fadvise_random] in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e151) 2023-10-18T19:23:24.887 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-10-18T19:23:24.889 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-10-18T19:23:24.900 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":6276,"stamp":"2023-10-18T19:23:24.680939+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527883816,"num_objects":380,"num_object_clones":1,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":159,"num_read":65589,"num_read_kb":133865,"num_write":18355,"num_write_kb":8833866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":67,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25684,"ondisk_log_size":25684,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1247204,"kb_used_data":318668,"kb_used_omap":0,"kb_used_meta":928512,"kb_avail":281868316,"statfs":{"total":289910292480,"available":288633155584,"internally_reserved":0,"allocated":326316032,"data_stored":324083503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":950796288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001638"},"pg_stats":[{"pgid":"2.7","version":"151'885","reported_seq":1731,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451392+0000","last_change":"2023-10-18T16:03:52.117247+0000","last_active":"2023-10-18T17:02:28.451392+0000","last_peered":"2023-10-18T17:02:28.451392+0000","last_clean":"2023-10-18T17:02:28.451392+0000","last_became_active":"2023-10-18T15:59:17.900985+0000","last_became_peered":"2023-10-18T15:59:17.900985+0000","last_unstale":"2023-10-18T17:02:28.451392+0000","last_undegraded":"2023-10-18T17:02:28.451392+0000","last_fullsized":"2023-10-18T17:02:28.451392+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":885,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47935573,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":557,"num_read_kb":221,"num_write":761,"num_write_kb":1107410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"151'943","reported_seq":1860,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451431+0000","last_change":"2023-10-18T16:03:52.117011+0000","last_active":"2023-10-18T17:02:28.451431+0000","last_peered":"2023-10-18T17:02:28.451431+0000","last_clean":"2023-10-18T17:02:28.451431+0000","last_became_active":"2023-10-18T15:59:17.899502+0000","last_became_peered":"2023-10-18T15:59:17.899502+0000","last_unstale":"2023-10-18T17:02:28.451431+0000","last_undegraded":"2023-10-18T17:02:28.451431+0000","last_fullsized":"2023-10-18T17:02:28.451431+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":943,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48419121,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":876,"num_write":771,"num_write_kb":1095242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"151'996","reported_seq":1883,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451519+0000","last_change":"2023-10-18T16:03:52.117176+0000","last_active":"2023-10-18T17:02:28.451519+0000","last_peered":"2023-10-18T17:02:28.451519+0000","last_clean":"2023-10-18T17:02:28.451519+0000","last_became_active":"2023-10-18T15:59:17.899385+0000","last_became_peered":"2023-10-18T15:59:17.899385+0000","last_unstale":"2023-10-18T17:02:28.451519+0000","last_undegraded":"2023-10-18T17:02:28.451519+0000","last_fullsized":"2023-10-18T17:02:28.451519+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":996,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61345877,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":1888,"num_write":888,"num_write_kb":1086061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"151'3171","reported_seq":8685,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451975+0000","last_change":"2023-10-18T16:03:52.126849+0000","last_active":"2023-10-18T17:02:28.451975+0000","last_peered":"2023-10-18T17:02:28.451975+0000","last_clean":"2023-10-18T17:02:28.451975+0000","last_became_active":"2023-10-18T15:59:25.951214+0000","last_became_peered":"2023-10-18T15:59:25.951214+0000","last_unstale":"2023-10-18T17:02:28.451975+0000","last_undegraded":"2023-10-18T17:02:28.451975+0000","last_fullsized":"2023-10-18T17:02:28.451975+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":3171,"ondisk_log_size":3171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11993788,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":42,"num_read":11681,"num_read_kb":35040,"num_write":1781,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":700,"num_flush":192,"num_flush_kb":4113,"num_evict":481,"num_evict_kb":15628,"num_promote":526,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"154'1382","reported_seq":2203,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:25.706779+0000","last_change":"2023-10-18T19:06:25.706779+0000","last_active":"2023-10-18T19:06:25.706779+0000","last_peered":"2023-10-18T19:06:25.706779+0000","last_clean":"2023-10-18T19:06:25.706779+0000","last_became_active":"2023-10-18T15:59:17.898490+0000","last_became_peered":"2023-10-18T15:59:17.898490+0000","last_unstale":"2023-10-18T19:06:25.706779+0000","last_undegraded":"2023-10-18T19:06:25.706779+0000","last_fullsized":"2023-10-18T19:06:25.706779+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1382","last_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_deep_scrub":"154'1382","last_deep_scrub_stamp":"2023-10-18T19:06:25.706688+0000","last_clean_scrub_stamp":"2023-10-18T19:06:25.706688+0000","log_size":1382,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56762553,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":525,"num_read_kb":1749,"num_write":791,"num_write_kb":1046876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"151'4208","reported_seq":11817,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451495+0000","last_change":"2023-10-18T16:03:52.119844+0000","last_active":"2023-10-18T17:02:28.451495+0000","last_peered":"2023-10-18T17:02:28.451495+0000","last_clean":"2023-10-18T17:02:28.451495+0000","last_became_active":"2023-10-18T15:59:25.950375+0000","last_became_peered":"2023-10-18T15:59:25.950375+0000","last_unstale":"2023-10-18T17:02:28.451495+0000","last_undegraded":"2023-10-18T17:02:28.451495+0000","last_fullsized":"2023-10-18T17:02:28.451495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4208,"ondisk_log_size":4208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36582279,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":36,"num_read":15606,"num_read_kb":18241,"num_write":2254,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":868,"num_flush":215,"num_flush_kb":14375,"num_evict":543,"num_evict_kb":20067,"num_promote":594,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":156,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:08:51.816230+0000","last_change":"2023-10-18T19:08:51.816230+0000","last_active":"2023-10-18T19:08:51.816230+0000","last_peered":"2023-10-18T19:08:51.816230+0000","last_clean":"2023-10-18T19:08:51.816230+0000","last_became_active":"2023-10-18T15:59:17.899873+0000","last_became_peered":"2023-10-18T15:59:17.899873+0000","last_unstale":"2023-10-18T19:08:51.816230+0000","last_undegraded":"2023-10-18T19:08:51.816230+0000","last_fullsized":"2023-10-18T19:08:51.816230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T19:08:51.816134+0000","last_clean_scrub_stamp":"2023-10-18T19:08:51.816134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"151'1352","reported_seq":2174,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451382+0000","last_change":"2023-10-18T16:03:52.116758+0000","last_active":"2023-10-18T17:02:28.451382+0000","last_peered":"2023-10-18T17:02:28.451382+0000","last_clean":"2023-10-18T17:02:28.451382+0000","last_became_active":"2023-10-18T15:59:17.897418+0000","last_became_peered":"2023-10-18T15:59:17.897418+0000","last_unstale":"2023-10-18T17:02:28.451382+0000","last_undegraded":"2023-10-18T17:02:28.451382+0000","last_fullsized":"2023-10-18T17:02:28.451382+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1352,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53129556,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":473,"num_read_kb":320,"num_write":796,"num_write_kb":1127272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"151'5063","reported_seq":11805,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451375+0000","last_change":"2023-10-18T16:03:52.117382+0000","last_active":"2023-10-18T17:02:28.451375+0000","last_peered":"2023-10-18T17:02:28.451375+0000","last_clean":"2023-10-18T17:02:28.451375+0000","last_became_active":"2023-10-18T15:59:25.951099+0000","last_became_peered":"2023-10-18T15:59:25.951099+0000","last_unstale":"2023-10-18T17:02:28.451375+0000","last_undegraded":"2023-10-18T17:02:28.451375+0000","last_fullsized":"2023-10-18T17:02:28.451375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":5063,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3872044,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":45,"num_read":16060,"num_read_kb":26177,"num_write":3275,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1218,"num_flush":198,"num_flush_kb":30,"num_evict":464,"num_evict_kb":19164,"num_promote":513,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"154'1016","reported_seq":1856,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-18T19:06:30.697726+0000","last_change":"2023-10-18T19:06:30.697726+0000","last_active":"2023-10-18T19:06:30.697726+0000","last_peered":"2023-10-18T19:06:30.697726+0000","last_clean":"2023-10-18T19:06:30.697726+0000","last_became_active":"2023-10-18T15:59:17.898505+0000","last_became_peered":"2023-10-18T15:59:17.898505+0000","last_unstale":"2023-10-18T19:06:30.697726+0000","last_undegraded":"2023-10-18T19:06:30.697726+0000","last_fullsized":"2023-10-18T19:06:30.697726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'1016","last_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_deep_scrub":"154'1016","last_deep_scrub_stamp":"2023-10-18T19:06:30.697632+0000","last_clean_scrub_stamp":"2023-10-18T19:06:30.697632+0000","log_size":1016,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84544329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":1,"num_read":557,"num_read_kb":2689,"num_write":858,"num_write_kb":1087549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":370,"num_omap_keys":40,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"151'4731","reported_seq":10972,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.452154+0000","last_change":"2023-10-18T16:03:52.127113+0000","last_active":"2023-10-18T17:02:28.452154+0000","last_peered":"2023-10-18T17:02:28.452154+0000","last_clean":"2023-10-18T17:02:28.452154+0000","last_became_active":"2023-10-18T15:59:25.951232+0000","last_became_peered":"2023-10-18T15:59:25.951232+0000","last_unstale":"2023-10-18T17:02:28.452154+0000","last_undegraded":"2023-10-18T17:02:28.452154+0000","last_fullsized":"2023-10-18T17:02:28.452154+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:24.940406+0000","last_clean_scrub_stamp":"2023-10-18T15:59:24.940406+0000","log_size":4731,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36029075,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":35,"num_read":17293,"num_read_kb":38084,"num_write":4556,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":659,"num_flush":195,"num_flush_kb":17,"num_evict":360,"num_evict_kb":20662,"num_promote":407,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"151'1014","reported_seq":1995,"reported_epoch":154,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-18T19:08:52.418700+0000","last_change":"2023-10-18T19:08:52.418700+0000","last_active":"2023-10-18T19:08:52.418700+0000","last_peered":"2023-10-18T19:08:52.418700+0000","last_clean":"2023-10-18T19:08:52.418700+0000","last_became_active":"2023-10-18T15:59:17.897919+0000","last_became_peered":"2023-10-18T15:59:17.897919+0000","last_unstale":"2023-10-18T19:08:52.418700+0000","last_undegraded":"2023-10-18T19:08:52.418700+0000","last_fullsized":"2023-10-18T19:08:52.418700+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":1014,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50577462,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":743,"num_read_kb":2693,"num_write":857,"num_write_kb":1122377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"151'923","reported_seq":1895,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-18T17:02:28.451435+0000","last_change":"2023-10-18T16:03:52.116900+0000","last_active":"2023-10-18T17:02:28.451435+0000","last_peered":"2023-10-18T17:02:28.451435+0000","last_clean":"2023-10-18T17:02:28.451435+0000","last_became_active":"2023-10-18T15:59:17.899341+0000","last_became_peered":"2023-10-18T15:59:17.899341+0000","last_unstale":"2023-10-18T17:02:28.451435+0000","last_undegraded":"2023-10-18T17:02:28.451435+0000","last_fullsized":"2023-10-18T17:02:28.451435+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T15:59:16.885414+0000","last_clean_scrub_stamp":"2023-10-18T15:59:16.885414+0000","log_size":923,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36692159,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":747,"num_read_kb":5887,"num_write":767,"num_write_kb":1153600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":88477186,"num_objects":205,"num_object_clones":0,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":158,"num_read":60640,"num_read_kb":117542,"num_write":11866,"num_write_kb":7479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":15,"num_bytes_hit_set_archive":3445,"num_flush":800,"num_flush_kb":18535,"num_evict":1848,"num_evict_kb":75521,"num_promote":2040,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97886208,"data_stored":97737732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17173,"ondisk_log_size":17173,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":439406630,"num_objects":175,"num_object_clones":1,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":1,"num_read":4949,"num_read_kb":16323,"num_write":6489,"num_write_kb":8826387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":593,"num_omap_keys":65,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224534528,"data_stored":224305228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8511,"ondisk_log_size":8511,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740822,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409604,"kb_used_data":100092,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93962236,"statfs":{"total":96636764160,"available":96217329664,"internally_reserved":0,"allocated":102494208,"data_stored":101755981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:22:59 2023","interfaces":[{"interface":"back","average":{"1min":0.219,"5min":0.23200000000000001,"15min":0.224},"min":{"1min":0.155,"5min":0.13900000000000001,"15min":0.13900000000000001},"max":{"1min":0.34999999999999998,"5min":0.438,"15min":0.47599999999999998},"last":0.247},{"interface":"front","average":{"1min":0.26000000000000001,"5min":0.27500000000000002,"15min":0.26600000000000001},"min":{"1min":0.21299999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.39500000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.27100000000000002}]},{"osd":1,"last update":"Wed Oct 18 19:22:59 2023","interfaces":[{"interface":"back","average":{"1min":0.23499999999999999,"5min":0.24099999999999999,"15min":0.23499999999999999},"min":{"1min":0.183,"5min":0.17799999999999999,"15min":0.157},"max":{"1min":0.38,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.183},{"interface":"front","average":{"1min":0.24299999999999999,"5min":0.253,"15min":0.246},"min":{"1min":0.19400000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.35899999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.21199999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740823,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395012,"kb_used_data":85500,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93976828,"statfs":{"total":96636764160,"available":96232271872,"internally_reserved":0,"allocated":87552000,"data_stored":86841620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 19:22:58 2023","interfaces":[{"interface":"back","average":{"1min":0.23799999999999999,"5min":0.219,"15min":0.218},"min":{"1min":0.14899999999999999,"5min":0.14899999999999999,"15min":0.13700000000000001},"max":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.47799999999999998},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.246,"5min":0.23200000000000001,"15min":0.23400000000000001},"min":{"1min":0.182,"5min":0.17799999999999999,"15min":0.156},"max":{"1min":0.443,"5min":0.443,"15min":0.48399999999999999},"last":0.19500000000000001}]},{"osd":2,"last update":"Wed Oct 18 19:22:58 2023","interfaces":[{"interface":"back","average":{"1min":0.249,"5min":0.23499999999999999,"15min":0.23699999999999999},"min":{"1min":0.17199999999999999,"5min":0.16700000000000001,"15min":0.152},"max":{"1min":0.435,"5min":0.435,"15min":0.44600000000000001},"last":0.20599999999999999},{"interface":"front","average":{"1min":0.27200000000000002,"5min":0.26200000000000001,"15min":0.25800000000000001},"min":{"1min":0.193,"5min":0.188,"15min":0.16600000000000001},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.54600000000000004},"last":0.251}]}]},{"osd":1,"up_from":8,"seq":34359740824,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442588,"kb_used_data":133076,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93929252,"statfs":{"total":96636764160,"available":96183554048,"internally_reserved":0,"allocated":136269824,"data_stored":135485902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 19:22:53 2023","interfaces":[{"interface":"back","average":{"1min":0.21199999999999999,"5min":0.215,"15min":0.22},"min":{"1min":0.14799999999999999,"5min":0.14399999999999999,"15min":0.14399999999999999},"max":{"1min":0.38800000000000001,"5min":0.42199999999999999,"15min":0.46200000000000002},"last":0.19500000000000001},{"interface":"front","average":{"1min":0.248,"5min":0.25600000000000001,"15min":0.254},"min":{"1min":0.17699999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.432,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.23599999999999999}]},{"osd":2,"last update":"Wed Oct 18 19:22:53 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.221,"15min":0.23000000000000001},"min":{"1min":0.161,"5min":0.161,"15min":0.154},"max":{"1min":0.439,"5min":0.439,"15min":0.44700000000000001},"last":0.249},{"interface":"front","average":{"1min":0.25900000000000001,"5min":0.26100000000000001,"15min":0.26600000000000001},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.41599999999999998,"5min":0.48599999999999999,"15min":0.48899999999999999},"last":0.219}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60014592,"data_stored":59945680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112267264,"data_stored":112152614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52252672,"data_stored":52206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26238976,"data_stored":26215759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22704128,"data_stored":22653107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48943104,"data_stored":48868866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T19:23:24.903 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:23:24.903 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:23:24.904 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:23:24.904 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:23:24.904 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:23:24.905 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:23:24.905 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:23:24.906 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-18T15:59:24.940406+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=24, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:23:24.906 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:23:24.907 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T15:59:16.885414+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=15, tm_min=59, tm_sec=16, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=19, tm_min=2, tm_sec=23, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T19:23:24.907 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph-c_84698e743199450ee5f6969d22ebe6bed3a68e4a/qa/tasks/ceph.py", line 1917, in task osd_scrub_pgs(ctx, config) File "/home/teuthworker/src/github.com_ceph_ceph-c_84698e743199450ee5f6969d22ebe6bed3a68e4a/qa/tasks/ceph.py", line 1312, in osd_scrub_pgs raise RuntimeError('Exiting scrub checking -- not all pgs scrubbed.') RuntimeError: Exiting scrub checking -- not all pgs scrubbed. 2023-10-18T19:23:24.916 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-18T19:23:24.916 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-18T19:23:24.917 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-18T19:23:24.917 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T19:23:24.949 INFO:tasks.ceph.osd.0:Stopped 2023-10-18T19:23:24.950 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-18T19:23:24.951 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T19:23:25.024 INFO:tasks.ceph.osd.1:Stopped 2023-10-18T19:23:25.025 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-18T19:23:25.025 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T19:23:25.098 INFO:tasks.ceph.osd.2:Stopped 2023-10-18T19:23:25.099 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-18T19:23:25.100 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-18T19:23:25.100 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T19:23:25.161 INFO:tasks.ceph.mgr.x:Stopped 2023-10-18T19:23:25.162 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-18T19:23:25.162 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-18T19:23:25.163 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T19:23:25.238 INFO:tasks.ceph.mon.a:Stopped 2023-10-18T19:23:25.238 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-18T19:23:25.239 DEBUG:teuthology.orchestra.run.smithi082:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | head -n 1 2023-10-18T19:23:25.302 INFO:teuthology.orchestra.run.smithi082.stdout:2023-10-18T16:04:39.550673+0000 osd.0 (osd.0) 3 : cluster [WRN] 1 slow requests (by type [ 'reached pg' : 1 ] most affected pool [ 'cache' : 1 ]) 2023-10-18T19:23:25.304 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-18T19:23:25.305 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi082.front.sepia.ceph.com 2023-10-18T19:23:25.305 DEBUG:teuthology.orchestra.run.smithi082:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-18T19:23:25.807 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi082.front.sepia.ceph.com 2023-10-18T19:23:25.807 DEBUG:teuthology.orchestra.run.smithi082:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-18T19:23:25.939 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi082.front.sepia.ceph.com 2023-10-18T19:23:25.939 DEBUG:teuthology.orchestra.run.smithi082:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-18T19:23:26.019 INFO:tasks.ceph:Archiving mon data... 2023-10-18T19:23:26.021 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-17_20:38:29-rbd-wip-yuri2-testing-2023-10-11-0853-pacific-distro-default-smithi/7431146/data/mon.a.tgz 2023-10-18T19:23:26.022 DEBUG:teuthology.orchestra.run.smithi082:> mktemp 2023-10-18T19:23:26.037 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/tmp.wKMMQCMDpu 2023-10-18T19:23:26.038 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.wKMMQCMDpu 2023-10-18T19:23:26.302 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0666 /tmp/tmp.wKMMQCMDpu 2023-10-18T19:23:26.383 DEBUG:teuthology.orchestra.remote:smithi082:/tmp/tmp.wKMMQCMDpu is 613KB 2023-10-18T19:23:26.483 DEBUG:teuthology.orchestra.run.smithi082:> rm -fr /tmp/tmp.wKMMQCMDpu 2023-10-18T19:23:26.497 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-18T19:23:26.497 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-18T19:23:26.561 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-18T19:23:26.561 DEBUG:teuthology.orchestra.run.smithi082:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-18T19:23:26.625 INFO:teuthology.orchestra.run.smithi082.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-18T19:23:26.631 INFO:tasks.ceph:Archiving crash dumps... 2023-10-18T19:23:26.633 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-17_20:38:29-rbd-wip-yuri2-testing-2023-10-11-0853-pacific-distro-default-smithi/7431146/remote/smithi082/crash 2023-10-18T19:23:26.634 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-18T19:23:26.694 INFO:tasks.ceph:Compressing logs... 2023-10-18T19:23:26.695 DEBUG:teuthology.orchestra.run.smithi082:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-18T19:23:27.450 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106289.log 2023-10-18T19:23:27.450 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114399.log 2023-10-18T19:23:27.451 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103485.log 2023-10-18T19:23:27.451 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106289.log.gz 2023-10-18T19:23:27.451 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111697.log 2023-10-18T19:23:27.452 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114399.log.gz 2023-10-18T19:23:27.452 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129694.log 2023-10-18T19:23:27.452 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103485.log.gz 2023-10-18T19:23:27.452 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135499.log 2023-10-18T19:23:27.453 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111697.log.gz 2023-10-18T19:23:27.453 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119398.log 2023-10-18T19:23:27.453 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129694.log.gz 2023-10-18T19:23:27.454 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122111.log 2023-10-18T19:23:27.454 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135499.log.gz 2023-10-18T19:23:27.454 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103128.log 2023-10-18T19:23:27.454 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119398.log.gz 2023-10-18T19:23:27.455 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90526.log 2023-10-18T19:23:27.455 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122111.log.gz 2023-10-18T19:23:27.455 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100215.log 2023-10-18T19:23:27.455 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103128.log.gz 2023-10-18T19:23:27.456 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.90526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94635.log 2023-10-18T19:23:27.456 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90526.log.gz 2023-10-18T19:23:27.456 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134199.log 2023-10-18T19:23:27.457 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100215.log.gz 2023-10-18T19:23:27.457 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113382.log 2023-10-18T19:23:27.457 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94635.log.gz 2023-10-18T19:23:27.457 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98955.log 2023-10-18T19:23:27.458 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134199.log.gz 2023-10-18T19:23:27.458 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117592.log 2023-10-18T19:23:27.458 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113382.log.gz 2023-10-18T19:23:27.459 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91356.log 2023-10-18T19:23:27.459 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98955.log.gz 2023-10-18T19:23:27.459 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.117592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98215.log 2023-10-18T19:23:27.460 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117592.log.gz 2023-10-18T19:23:27.460 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112513.log 2023-10-18T19:23:27.460 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91356.log.gz 2023-10-18T19:23:27.461 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126045.log 2023-10-18T19:23:27.461 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.98215.log.gz 2023-10-18T19:23:27.461 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137659.log 2023-10-18T19:23:27.462 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112513.log.gz 2023-10-18T19:23:27.462 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133732.log 2023-10-18T19:23:27.462 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126045.log.gz 2023-10-18T19:23:27.462 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.137659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124217.log 2023-10-18T19:23:27.463 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.137659.log.gz 2023-10-18T19:23:27.463 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131582.log 2023-10-18T19:23:27.463 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133732.log.gz 2023-10-18T19:23:27.464 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101024.log 2023-10-18T19:23:27.464 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124217.log.gz 2023-10-18T19:23:27.464 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110678.log 2023-10-18T19:23:27.464 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131582.log.gz 2023-10-18T19:23:27.465 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122897.log 2023-10-18T19:23:27.465 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101024.log.gz 2023-10-18T19:23:27.465 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134135.log 2023-10-18T19:23:27.465 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110678.log.gz 2023-10-18T19:23:27.466 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96829.log 2023-10-18T19:23:27.466 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122897.log.gz 2023-10-18T19:23:27.466 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98895.log 2023-10-18T19:23:27.467 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134135.log.gz 2023-10-18T19:23:27.467 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94206.log 2023-10-18T19:23:27.467 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96829.log.gz 2023-10-18T19:23:27.467 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92025.log 2023-10-18T19:23:27.468 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98895.log.gz 2023-10-18T19:23:27.468 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110377.log 2023-10-18T19:23:27.468 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94206.log.gz 2023-10-18T19:23:27.468 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123532.log 2023-10-18T19:23:27.469 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92025.log.gz 2023-10-18T19:23:27.469 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118518.log 2023-10-18T19:23:27.469 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110377.log.gz 2023-10-18T19:23:27.470 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132764.log 2023-10-18T19:23:27.470 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123532.log.gz 2023-10-18T19:23:27.470 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103569.log 2023-10-18T19:23:27.471 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118518.log.gz 2023-10-18T19:23:27.471 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107332.log 2023-10-18T19:23:27.471 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132764.log.gz 2023-10-18T19:23:27.471 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129754.log 2023-10-18T19:23:27.472 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103569.log.gz 2023-10-18T19:23:27.472 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93916.log 2023-10-18T19:23:27.472 INFO:teuthology.orchestra.run.smithi082.stderr: 15.7% -- replaced with /var/log/ceph/ceph-client.admin.107332.log.gz 2023-10-18T19:23:27.473 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91850.log 2023-10-18T19:23:27.473 INFO:teuthology.orchestra.run.smithi082.stderr: 53.0% -- replaced with /var/log/ceph/ceph-client.admin.129754.log.gz 2023-10-18T19:23:27.473 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112713.log 2023-10-18T19:23:27.473 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.93916.log.gz 2023-10-18T19:23:27.474 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100475.log 2023-10-18T19:23:27.474 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91850.log.gz 2023-10-18T19:23:27.474 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93069.log 2023-10-18T19:23:27.474 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112713.log.gz 2023-10-18T19:23:27.475 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106871.log 2023-10-18T19:23:27.475 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100475.log.gz 2023-10-18T19:23:27.475 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135048.log 2023-10-18T19:23:27.476 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93069.log.gz 2023-10-18T19:23:27.476 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123806.log 2023-10-18T19:23:27.476 INFO:teuthology.orchestra.run.smithi082.stderr: 22.8% -- replaced with /var/log/ceph/ceph-client.admin.106871.log.gz 2023-10-18T19:23:27.476 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99395.log 2023-10-18T19:23:27.477 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135048.log.gz 2023-10-18T19:23:27.477 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107854.log 2023-10-18T19:23:27.477 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123806.log.gz 2023-10-18T19:23:27.478 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109070.log 2023-10-18T19:23:27.478 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99395.log.gz 2023-10-18T19:23:27.478 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110201.log 2023-10-18T19:23:27.478 INFO:teuthology.orchestra.run.smithi082.stderr: 56.3% -- replaced with /var/log/ceph/ceph-client.admin.107854.log.gz 2023-10-18T19:23:27.479 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142694.log 2023-10-18T19:23:27.479 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109070.log.gz 2023-10-18T19:23:27.479 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89980.log 2023-10-18T19:23:27.479 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110201.log.gz 2023-10-18T19:23:27.480 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.142694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142000.log 2023-10-18T19:23:27.480 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.142694.log.gz 2023-10-18T19:23:27.480 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.89980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101823.log 2023-10-18T19:23:27.481 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89980.log.gz 2023-10-18T19:23:27.481 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.142000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100155.log 2023-10-18T19:23:27.481 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.142000.log.gz 2023-10-18T19:23:27.482 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94549.log 2023-10-18T19:23:27.482 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101823.log.gz 2023-10-18T19:23:27.482 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137701.log 2023-10-18T19:23:27.482 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100155.log.gz 2023-10-18T19:23:27.483 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127601.log 2023-10-18T19:23:27.483 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94549.log.gz 2023-10-18T19:23:27.483 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.137701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133022.log 2023-10-18T19:23:27.484 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.137701.log.gz 2023-10-18T19:23:27.484 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121638.log 2023-10-18T19:23:27.484 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127601.log.gz 2023-10-18T19:23:27.484 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133516.log 2023-10-18T19:23:27.485 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133022.log.gz 2023-10-18T19:23:27.485 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96314.log 2023-10-18T19:23:27.485 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121638.log.gz 2023-10-18T19:23:27.486 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129735.log 2023-10-18T19:23:27.486 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133516.log.gz 2023-10-18T19:23:27.486 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108548.log 2023-10-18T19:23:27.486 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96314.log.gz 2023-10-18T19:23:27.487 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126979.log 2023-10-18T19:23:27.487 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129735.log.gz 2023-10-18T19:23:27.487 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114334.log 2023-10-18T19:23:27.487 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108548.log.gz 2023-10-18T19:23:27.488 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101313.log 2023-10-18T19:23:27.488 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126979.log.gz 2023-10-18T19:23:27.488 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109728.log 2023-10-18T19:23:27.489 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114334.log.gz 2023-10-18T19:23:27.489 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104559.log 2023-10-18T19:23:27.489 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101313.log.gz 2023-10-18T19:23:27.489 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119912.log 2023-10-18T19:23:27.490 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109728.log.gz 2023-10-18T19:23:27.490 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143080.log 2023-10-18T19:23:27.490 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104559.log.gz 2023-10-18T19:23:27.490 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126646.log 2023-10-18T19:23:27.491 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119912.log.gz 2023-10-18T19:23:27.491 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.143080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108190.log 2023-10-18T19:23:27.491 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.143080.log.gz 2023-10-18T19:23:27.492 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122256.log 2023-10-18T19:23:27.492 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126646.log.gz 2023-10-18T19:23:27.492 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129614.log 2023-10-18T19:23:27.493 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108190.log.gz 2023-10-18T19:23:27.493 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97739.log 2023-10-18T19:23:27.493 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122256.log.gz 2023-10-18T19:23:27.494 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96743.log 2023-10-18T19:23:27.494 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129614.log.gz 2023-10-18T19:23:27.494 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106332.log 2023-10-18T19:23:27.494 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97739.log.gz 2023-10-18T19:23:27.495 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102146.log 2023-10-18T19:23:27.495 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96743.log.gz 2023-10-18T19:23:27.495 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140665.log 2023-10-18T19:23:27.496 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106332.log.gz 2023-10-18T19:23:27.496 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96142.log 2023-10-18T19:23:27.496 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102146.log.gz 2023-10-18T19:23:27.496 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.140665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118194.log 2023-10-18T19:23:27.497 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.140665.log.gz 2023-10-18T19:23:27.497 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96142.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96142.log.gz 2023-10-18T19:23:27.497 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113112.log 2023-10-18T19:23:27.498 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121939.log 2023-10-18T19:23:27.498 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118194.log.gz 2023-10-18T19:23:27.498 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109621.log 2023-10-18T19:23:27.498 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113112.log.gz 2023-10-18T19:23:27.499 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100888.log 2023-10-18T19:23:27.499 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121939.log.gz 2023-10-18T19:23:27.499 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134371.log 2023-10-18T19:23:27.499 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109621.log.gz 2023-10-18T19:23:27.500 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122638.log 2023-10-18T19:23:27.500 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100888.log.gz 2023-10-18T19:23:27.500 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128290.log 2023-10-18T19:23:27.501 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134371.log.gz 2023-10-18T19:23:27.501 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134393.log 2023-10-18T19:23:27.501 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122638.log.gz 2023-10-18T19:23:27.501 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99275.log 2023-10-18T19:23:27.502 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128290.log.gz 2023-10-18T19:23:27.502 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92608.log 2023-10-18T19:23:27.502 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134393.log.gz 2023-10-18T19:23:27.503 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99195.log 2023-10-18T19:23:27.503 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99275.log.gz 2023-10-18T19:23:27.503 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113463.log 2023-10-18T19:23:27.504 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.92608.log.gz 2023-10-18T19:23:27.504 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118780.log 2023-10-18T19:23:27.504 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99195.log.gz 2023-10-18T19:23:27.504 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106586.log 2023-10-18T19:23:27.505 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113463.log.gz 2023-10-18T19:23:27.505 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106913.log 2023-10-18T19:23:27.505 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118780.log.gz 2023-10-18T19:23:27.506 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94742.log 2023-10-18T19:23:27.506 INFO:teuthology.orchestra.run.smithi082.stderr: 17.8% -- replaced with /var/log/ceph/ceph-client.admin.106586.log.gz 2023-10-18T19:23:27.506 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102248.log 2023-10-18T19:23:27.506 INFO:teuthology.orchestra.run.smithi082.stderr: 18.5% -- replaced with /var/log/ceph/ceph-client.admin.106913.log.gz 2023-10-18T19:23:27.507 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94742.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94742.log.gz 2023-10-18T19:23:27.507 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132936.log 2023-10-18T19:23:27.507 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100455.log 2023-10-18T19:23:27.508 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102248.log.gz 2023-10-18T19:23:27.508 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111022.log 2023-10-18T19:23:27.508 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132936.log.gz 2023-10-18T19:23:27.508 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139935.log 2023-10-18T19:23:27.509 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100455.log.gz 2023-10-18T19:23:27.509 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115066.log 2023-10-18T19:23:27.509 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111022.log.gz 2023-10-18T19:23:27.510 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.139935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120816.log 2023-10-18T19:23:27.510 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.139935.log.gz 2023-10-18T19:23:27.510 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93456.log 2023-10-18T19:23:27.510 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115066.log.gz 2023-10-18T19:23:27.511 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94699.log 2023-10-18T19:23:27.511 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120816.log.gz 2023-10-18T19:23:27.511 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114205.log 2023-10-18T19:23:27.512 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93456.log.gz 2023-10-18T19:23:27.512 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105744.log 2023-10-18T19:23:27.512 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94699.log.gz 2023-10-18T19:23:27.513 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132485.log 2023-10-18T19:23:27.513 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114205.log.gz 2023-10-18T19:23:27.513 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123456.log 2023-10-18T19:23:27.514 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105744.log.gz 2023-10-18T19:23:27.514 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130387.log 2023-10-18T19:23:27.514 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132485.log.gz 2023-10-18T19:23:27.514 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107651.log 2023-10-18T19:23:27.515 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123456.log.gz 2023-10-18T19:23:27.515 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100616.log 2023-10-18T19:23:27.515 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130387.log.gz 2023-10-18T19:23:27.516 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96335.log 2023-10-18T19:23:27.516 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107651.log.gz 2023-10-18T19:23:27.516 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123284.log 2023-10-18T19:23:27.516 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100616.log.gz 2023-10-18T19:23:27.517 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97909.log 2023-10-18T19:23:27.517 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96335.log.gz 2023-10-18T19:23:27.517 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129381.log 2023-10-18T19:23:27.517 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123284.log.gz 2023-10-18T19:23:27.518 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122296.log 2023-10-18T19:23:27.518 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.97909.log.gz 2023-10-18T19:23:27.518 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123069.log 2023-10-18T19:23:27.519 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129381.log.gz 2023-10-18T19:23:27.519 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135155.log 2023-10-18T19:23:27.519 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122296.log.gz 2023-10-18T19:23:27.519 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104967.log 2023-10-18T19:23:27.520 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123069.log.gz 2023-10-18T19:23:27.520 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117700.log 2023-10-18T19:23:27.520 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135155.log.gz 2023-10-18T19:23:27.520 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126088.log 2023-10-18T19:23:27.521 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104967.log.gz 2023-10-18T19:23:27.521 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.117700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129969.log 2023-10-18T19:23:27.521 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117700.log.gz 2023-10-18T19:23:27.522 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128688.log 2023-10-18T19:23:27.522 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126088.log.gz 2023-10-18T19:23:27.522 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131084.log 2023-10-18T19:23:27.523 INFO:teuthology.orchestra.run.smithi082.stderr: 55.0% -- replaced with /var/log/ceph/ceph-client.admin.129969.log.gz 2023-10-18T19:23:27.523 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115590.log 2023-10-18T19:23:27.523 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128688.log.gz 2023-10-18T19:23:27.524 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120020.log 2023-10-18T19:23:27.524 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131084.log.gz 2023-10-18T19:23:27.524 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109228.log 2023-10-18T19:23:27.524 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115590.log.gz 2023-10-18T19:23:27.525 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141606.log 2023-10-18T19:23:27.525 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120020.log.gz 2023-10-18T19:23:27.525 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93933.log 2023-10-18T19:23:27.525 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109228.log.gz 2023-10-18T19:23:27.526 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.141606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98198.log 2023-10-18T19:23:27.526 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.141606.log.gz 2023-10-18T19:23:27.526 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121234.log 2023-10-18T19:23:27.527 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93933.log.gz 2023-10-18T19:23:27.527 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127349.log 2023-10-18T19:23:27.527 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.98198.log.gz 2023-10-18T19:23:27.528 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91764.log 2023-10-18T19:23:27.528 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121234.log.gz 2023-10-18T19:23:27.529 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91227.log 2023-10-18T19:23:27.529 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127349.log.gz 2023-10-18T19:23:27.529 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132399.log 2023-10-18T19:23:27.530 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91764.log.gz 2023-10-18T19:23:27.530 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107732.log 2023-10-18T19:23:27.531 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91227.log.gz 2023-10-18T19:23:27.531 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102834.log 2023-10-18T19:23:27.532 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132399.log.gz 2023-10-18T19:23:27.532 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96722.log 2023-10-18T19:23:27.533 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107732.log.gz 2023-10-18T19:23:27.533 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130753.log 2023-10-18T19:23:27.534 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102834.log.gz 2023-10-18T19:23:27.534 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141186.log 2023-10-18T19:23:27.535 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96722.log.gz 2023-10-18T19:23:27.535 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142305.log 2023-10-18T19:23:27.536 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130753.log.gz 2023-10-18T19:23:27.536 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.141186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91399.log 2023-10-18T19:23:27.536 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.141186.log.gz 2023-10-18T19:23:27.536 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.142305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92106.log 2023-10-18T19:23:27.537 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.142305.log.gz 2023-10-18T19:23:27.537 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103254.log 2023-10-18T19:23:27.537 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91399.log.gz 2023-10-18T19:23:27.538 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109986.log 2023-10-18T19:23:27.538 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92106.log.gz 2023-10-18T19:23:27.538 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103170.log 2023-10-18T19:23:27.538 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103254.log.gz 2023-10-18T19:23:27.539 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109965.log 2023-10-18T19:23:27.539 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109986.log.gz 2023-10-18T19:23:27.539 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114915.log 2023-10-18T19:23:27.539 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103170.log.gz 2023-10-18T19:23:27.540 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109965.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109965.log.gz 2023-10-18T19:23:27.540 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128988.log 2023-10-18T19:23:27.540 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99035.log 2023-10-18T19:23:27.541 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114915.log.gz 2023-10-18T19:23:27.541 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133920.log 2023-10-18T19:23:27.541 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128988.log.gz 2023-10-18T19:23:27.541 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95000.log 2023-10-18T19:23:27.542 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99035.log.gz 2023-10-18T19:23:27.542 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132850.log 2023-10-18T19:23:27.542 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133920.log.gz 2023-10-18T19:23:27.543 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114076.log 2023-10-18T19:23:27.543 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95000.log.gz 2023-10-18T19:23:27.543 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114829.log 2023-10-18T19:23:27.543 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132850.log.gz 2023-10-18T19:23:27.544 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95172.log 2023-10-18T19:23:27.544 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114076.log.gz 2023-10-18T19:23:27.544 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119568.log 2023-10-18T19:23:27.545 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114829.log.gz 2023-10-18T19:23:27.545 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118355.log 2023-10-18T19:23:27.545 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95172.log.gz 2023-10-18T19:23:27.546 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124280.log 2023-10-18T19:23:27.546 INFO:teuthology.orchestra.run.smithi082.stderr: 23.2% -- replaced with /var/log/ceph/ceph-client.admin.119568.log.gz 2023-10-18T19:23:27.546 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133328.log 2023-10-18T19:23:27.546 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118355.log.gz 2023-10-18T19:23:27.547 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105830.log 2023-10-18T19:23:27.547 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124280.log.gz 2023-10-18T19:23:27.547 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120730.log 2023-10-18T19:23:27.548 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133328.log.gz 2023-10-18T19:23:27.548 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126862.log 2023-10-18T19:23:27.548 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105830.log.gz 2023-10-18T19:23:27.548 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106397.log 2023-10-18T19:23:27.549 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120730.log.gz 2023-10-18T19:23:27.549 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118001.log 2023-10-18T19:23:27.549 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126862.log.gz 2023-10-18T19:23:27.549 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98875.log 2023-10-18T19:23:27.550 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106397.log.gz 2023-10-18T19:23:27.550 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108511.log 2023-10-18T19:23:27.550 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118001.log.gz 2023-10-18T19:23:27.551 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130538.log 2023-10-18T19:23:27.551 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98875.log.gz 2023-10-18T19:23:27.551 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94098.log 2023-10-18T19:23:27.551 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.108511.log.gz 2023-10-18T19:23:27.552 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138116.log 2023-10-18T19:23:27.552 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130538.log.gz 2023-10-18T19:23:27.552 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111156.log 2023-10-18T19:23:27.552 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94098.log.gz 2023-10-18T19:23:27.553 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.138116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109049.log 2023-10-18T19:23:27.553 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.138116.log.gz 2023-10-18T19:23:27.553 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111598.log 2023-10-18T19:23:27.554 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111156.log.gz 2023-10-18T19:23:27.554 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119238.log 2023-10-18T19:23:27.554 INFO:teuthology.orchestra.run.smithi082.stderr: 23.2% -- replaced with /var/log/ceph/ceph-client.admin.109049.log.gz 2023-10-18T19:23:27.555 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96529.log 2023-10-18T19:23:27.555 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111598.log.gz 2023-10-18T19:23:27.555 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129514.log 2023-10-18T19:23:27.555 INFO:teuthology.orchestra.run.smithi082.stderr: 23.3% -- replaced with /var/log/ceph/ceph-client.admin.119238.log.gz 2023-10-18T19:23:27.556 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104752.log 2023-10-18T19:23:27.556 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96529.log.gz 2023-10-18T19:23:27.556 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141522.log 2023-10-18T19:23:27.557 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129514.log.gz 2023-10-18T19:23:27.557 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127786.log 2023-10-18T19:23:27.557 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104752.log.gz 2023-10-18T19:23:27.557 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.141522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92532.log 2023-10-18T19:23:27.558 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.141522.log.gz 2023-10-18T19:23:27.558 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104688.log 2023-10-18T19:23:27.558 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127786.log.gz 2023-10-18T19:23:27.559 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119483.log 2023-10-18T19:23:27.559 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.92532.log.gz 2023-10-18T19:23:27.559 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122516.log 2023-10-18T19:23:27.559 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104688.log.gz 2023-10-18T19:23:27.560 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125809.log 2023-10-18T19:23:27.560 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119483.log.gz 2023-10-18T19:23:27.560 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98249.log 2023-10-18T19:23:27.561 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122516.log.gz 2023-10-18T19:23:27.561 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126264.log 2023-10-18T19:23:27.561 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125809.log.gz 2023-10-18T19:23:27.561 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128669.log 2023-10-18T19:23:27.562 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.98249.log.gz 2023-10-18T19:23:27.562 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128016.log 2023-10-18T19:23:27.562 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126264.log.gz 2023-10-18T19:23:27.563 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100395.log 2023-10-18T19:23:27.563 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128669.log.gz 2023-10-18T19:23:27.563 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131166.log 2023-10-18T19:23:27.563 INFO:teuthology.orchestra.run.smithi082.stderr: 50.9% -- replaced with /var/log/ceph/ceph-client.admin.128016.log.gz 2023-10-18T19:23:27.565 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140279.log 2023-10-18T19:23:27.565 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100395.log.gz 2023-10-18T19:23:27.566 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93649.log 2023-10-18T19:23:27.566 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131166.log.gz 2023-10-18T19:23:27.566 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.140279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123004.log 2023-10-18T19:23:27.567 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.140279.log.gz 2023-10-18T19:23:27.567 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143130.log 2023-10-18T19:23:27.567 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93649.log.gz 2023-10-18T19:23:27.567 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133080.log 2023-10-18T19:23:27.568 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123004.log.gz 2023-10-18T19:23:27.568 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.143130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94528.log 2023-10-18T19:23:27.568 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.143130.log.gz 2023-10-18T19:23:27.569 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92070.log 2023-10-18T19:23:27.569 INFO:teuthology.orchestra.run.smithi082.stderr: 12.1% -- replaced with /var/log/ceph/ceph-client.admin.133080.log.gz 2023-10-18T19:23:27.569 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91979.log 2023-10-18T19:23:27.569 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94528.log.gz 2023-10-18T19:23:27.570 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104094.log 2023-10-18T19:23:27.570 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92070.log.gz 2023-10-18T19:23:27.570 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115697.log 2023-10-18T19:23:27.571 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91979.log.gz 2023-10-18T19:23:27.571 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98096.log 2023-10-18T19:23:27.571 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104094.log.gz 2023-10-18T19:23:27.571 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92456.log 2023-10-18T19:23:27.572 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115697.log.gz 2023-10-18T19:23:27.572 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115826.log 2023-10-18T19:23:27.572 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.98096.log.gz 2023-10-18T19:23:27.572 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122538.log 2023-10-18T19:23:27.573 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.92456.log.gz 2023-10-18T19:23:27.573 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119113.log 2023-10-18T19:23:27.573 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115826.log.gz 2023-10-18T19:23:27.574 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135672.log 2023-10-18T19:23:27.574 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122538.log.gz 2023-10-18T19:23:27.574 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120494.log 2023-10-18T19:23:27.574 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119113.log.gz 2023-10-18T19:23:27.575 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129048.log 2023-10-18T19:23:27.575 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135672.log.gz 2023-10-18T19:23:27.575 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109793.log 2023-10-18T19:23:27.576 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120494.log.gz 2023-10-18T19:23:27.576 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117657.log 2023-10-18T19:23:27.576 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129048.log.gz 2023-10-18T19:23:27.577 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119419.log 2023-10-18T19:23:27.577 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109793.log.gz 2023-10-18T19:23:27.577 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.117657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127096.log 2023-10-18T19:23:27.577 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117657.log.gz 2023-10-18T19:23:27.578 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126435.log 2023-10-18T19:23:27.578 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119419.log.gz 2023-10-18T19:23:27.578 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126845.log 2023-10-18T19:23:27.578 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127096.log.gz 2023-10-18T19:23:27.579 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100175.log 2023-10-18T19:23:27.579 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126435.log.gz 2023-10-18T19:23:27.579 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99355.log 2023-10-18T19:23:27.580 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126845.log.gz 2023-10-18T19:23:27.580 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138460.log 2023-10-18T19:23:27.580 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100175.log.gz 2023-10-18T19:23:27.580 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134350.log 2023-10-18T19:23:27.581 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99355.log.gz 2023-10-18T19:23:27.581 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.138460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114786.log 2023-10-18T19:23:27.581 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.138460.log.gz 2023-10-18T19:23:27.581 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138586.log 2023-10-18T19:23:27.582 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134350.log.gz 2023-10-18T19:23:27.582 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106892.log 2023-10-18T19:23:27.582 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114786.log.gz 2023-10-18T19:23:27.583 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.138586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99815.log 2023-10-18T19:23:27.583 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.138586.log.gz 2023-10-18T19:23:27.583 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101160.log 2023-10-18T19:23:27.583 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106892.log.gz 2023-10-18T19:23:27.584 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107171.log 2023-10-18T19:23:27.584 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99815.log.gz 2023-10-18T19:23:27.584 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127504.log 2023-10-18T19:23:27.584 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101160.log.gz 2023-10-18T19:23:27.585 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123155.log 2023-10-18T19:23:27.585 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107171.log.gz 2023-10-18T19:23:27.585 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101279.log 2023-10-18T19:23:27.586 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127504.log.gz 2023-10-18T19:23:27.586 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94399.log 2023-10-18T19:23:27.587 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123155.log.gz 2023-10-18T19:23:27.587 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109328.log 2023-10-18T19:23:27.587 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101279.log.gz 2023-10-18T19:23:27.588 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105340.log 2023-10-18T19:23:27.588 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94399.log.gz 2023-10-18T19:23:27.588 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111976.log 2023-10-18T19:23:27.588 INFO:teuthology.orchestra.run.smithi082.stderr: 22.8% -- replaced with /var/log/ceph/ceph-client.admin.109328.log.gz 2023-10-18T19:23:27.589 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134049.log 2023-10-18T19:23:27.589 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105340.log.gz 2023-10-18T19:23:27.589 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132334.log 2023-10-18T19:23:27.589 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111976.log.gz 2023-10-18T19:23:27.590 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120983.log 2023-10-18T19:23:27.590 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134049.log.gz 2023-10-18T19:23:27.590 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102044.log 2023-10-18T19:23:27.591 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132334.log.gz 2023-10-18T19:23:27.591 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118108.log 2023-10-18T19:23:27.591 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120983.log.gz 2023-10-18T19:23:27.591 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95280.log 2023-10-18T19:23:27.592 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102044.log.gz 2023-10-18T19:23:27.592 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120472.log 2023-10-18T19:23:27.593 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118108.log.gz 2023-10-18T19:23:27.593 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132958.log 2023-10-18T19:23:27.593 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95280.log.gz 2023-10-18T19:23:27.593 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101432.log 2023-10-18T19:23:27.594 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120472.log.gz 2023-10-18T19:23:27.594 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127117.log 2023-10-18T19:23:27.594 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132958.log.gz 2023-10-18T19:23:27.595 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91528.log 2023-10-18T19:23:27.595 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101432.log.gz 2023-10-18T19:23:27.595 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121702.log 2023-10-18T19:23:27.595 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127117.log.gz 2023-10-18T19:23:27.596 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103863.log 2023-10-18T19:23:27.596 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91528.log.gz 2023-10-18T19:23:27.596 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134608.log 2023-10-18T19:23:27.596 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121702.log.gz 2023-10-18T19:23:27.597 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102095.log 2023-10-18T19:23:27.597 INFO:teuthology.orchestra.run.smithi082.stderr: 22.9% -- replaced with /var/log/ceph/ceph-client.admin.103863.log.gz 2023-10-18T19:23:27.598 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92324.log 2023-10-18T19:23:27.598 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134608.log.gz 2023-10-18T19:23:27.598 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94613.log 2023-10-18T19:23:27.598 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102095.log.gz 2023-10-18T19:23:27.599 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119547.log 2023-10-18T19:23:27.599 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92324.log.gz 2023-10-18T19:23:27.599 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122556.log 2023-10-18T19:23:27.600 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94613.log.gz 2023-10-18T19:23:27.600 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111209.log 2023-10-18T19:23:27.600 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119547.log.gz 2023-10-18T19:23:27.600 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93327.log 2023-10-18T19:23:27.601 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122556.log.gz 2023-10-18T19:23:27.601 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123090.log 2023-10-18T19:23:27.601 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111209.log.gz 2023-10-18T19:23:27.601 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91721.log 2023-10-18T19:23:27.602 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93327.log.gz 2023-10-18T19:23:27.602 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132678.log 2023-10-18T19:23:27.602 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123090.log.gz 2023-10-18T19:23:27.603 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98079.log 2023-10-18T19:23:27.603 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91721.log.gz 2023-10-18T19:23:27.603 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91893.log 2023-10-18T19:23:27.603 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132678.log.gz 2023-10-18T19:23:27.604 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97450.log 2023-10-18T19:23:27.604 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.98079.log.gz 2023-10-18T19:23:27.604 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101789.log 2023-10-18T19:23:27.604 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91893.log.gz 2023-10-18T19:23:27.605 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114593.log 2023-10-18T19:23:27.605 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97450.log.gz 2023-10-18T19:23:27.605 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142652.log 2023-10-18T19:23:27.606 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101789.log.gz 2023-10-18T19:23:27.606 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127703.log 2023-10-18T19:23:27.606 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114593.log.gz 2023-10-18T19:23:27.606 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.142652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118087.log 2023-10-18T19:23:27.607 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.142652.log.gz 2023-10-18T19:23:27.607 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139725.log 2023-10-18T19:23:27.607 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127703.log.gz 2023-10-18T19:23:27.608 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103401.log 2023-10-18T19:23:27.608 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118087.log.gz 2023-10-18T19:23:27.608 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.139725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115378.log 2023-10-18T19:23:27.609 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.139725.log.gz 2023-10-18T19:23:27.609 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139499.log 2023-10-18T19:23:27.609 INFO:teuthology.orchestra.run.smithi082.stderr: 22.8% -- replaced with /var/log/ceph/ceph-client.admin.103401.log.gz 2023-10-18T19:23:27.609 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121831.log 2023-10-18T19:23:27.610 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115378.log.gz 2023-10-18T19:23:27.610 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.139499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99935.log 2023-10-18T19:23:27.610 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.139499.log.gz 2023-10-18T19:23:27.611 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142862.log 2023-10-18T19:23:27.611 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121831.log.gz 2023-10-18T19:23:27.611 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94141.log 2023-10-18T19:23:27.611 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99935.log.gz 2023-10-18T19:23:27.612 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.142862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105873.log 2023-10-18T19:23:27.612 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.142862.log.gz 2023-10-18T19:23:27.612 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103506.log 2023-10-18T19:23:27.613 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94141.log.gz 2023-10-18T19:23:27.613 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107608.log 2023-10-18T19:23:27.613 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105873.log.gz 2023-10-18T19:23:27.613 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129028.log 2023-10-18T19:23:27.614 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103506.log.gz 2023-10-18T19:23:27.614 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132463.log 2023-10-18T19:23:27.614 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107608.log.gz 2023-10-18T19:23:27.615 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125336.log 2023-10-18T19:23:27.615 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129028.log.gz 2023-10-18T19:23:27.615 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118327.log 2023-10-18T19:23:27.615 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132463.log.gz 2023-10-18T19:23:27.616 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90696.log 2023-10-18T19:23:27.616 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125336.log.gz 2023-10-18T19:23:27.616 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122378.log 2023-10-18T19:23:27.617 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118327.log.gz 2023-10-18T19:23:27.617 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.90696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126454.log 2023-10-18T19:23:27.617 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90696.log.gz 2023-10-18T19:23:27.617 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141564.log 2023-10-18T19:23:27.618 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122378.log.gz 2023-10-18T19:23:27.618 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131308.log 2023-10-18T19:23:27.619 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126454.log.gz 2023-10-18T19:23:27.619 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.141564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106139.log 2023-10-18T19:23:27.619 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.141564.log.gz 2023-10-18T19:23:27.619 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133633.log 2023-10-18T19:23:27.620 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131308.log.gz 2023-10-18T19:23:27.620 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111675.log 2023-10-18T19:23:27.620 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106139.log.gz 2023-10-18T19:23:27.620 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95559.log 2023-10-18T19:23:27.621 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133633.log.gz 2023-10-18T19:23:27.621 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91463.log 2023-10-18T19:23:27.621 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111675.log.gz 2023-10-18T19:23:27.622 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92494.log 2023-10-18T19:23:27.622 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95559.log.gz 2023-10-18T19:23:27.622 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127408.log 2023-10-18T19:23:27.622 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91463.log.gz 2023-10-18T19:23:27.623 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108966.log 2023-10-18T19:23:27.623 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.92494.log.gz 2023-10-18T19:23:27.623 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100701.log 2023-10-18T19:23:27.623 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127408.log.gz 2023-10-18T19:23:27.624 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102771.log 2023-10-18T19:23:27.624 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108966.log.gz 2023-10-18T19:23:27.624 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102687.log 2023-10-18T19:23:27.625 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100701.log.gz 2023-10-18T19:23:27.625 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134854.log 2023-10-18T19:23:27.625 INFO:teuthology.orchestra.run.smithi082.stderr: 23.2% -- replaced with /var/log/ceph/ceph-client.admin.102771.log.gz 2023-10-18T19:23:27.625 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140833.log 2023-10-18T19:23:27.626 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102687.log.gz 2023-10-18T19:23:27.626 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129108.log 2023-10-18T19:23:27.626 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134854.log.gz 2023-10-18T19:23:27.627 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.140833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123687.log 2023-10-18T19:23:27.627 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.140833.log.gz 2023-10-18T19:23:27.627 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115044.log 2023-10-18T19:23:27.627 INFO:teuthology.orchestra.run.smithi082.stderr: 56.5% -- replaced with /var/log/ceph/ceph-client.admin.129108.log.gz 2023-10-18T19:23:27.628 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92646.log 2023-10-18T19:23:27.628 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123687.log.gz 2023-10-18T19:23:27.628 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92052.log 2023-10-18T19:23:27.629 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115044.log.gz 2023-10-18T19:23:27.629 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98436.log 2023-10-18T19:23:27.629 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92646.log.gz 2023-10-18T19:23:27.630 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116084.log 2023-10-18T19:23:27.630 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92052.log.gz 2023-10-18T19:23:27.630 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125228.log 2023-10-18T19:23:27.630 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.98436.log.gz 2023-10-18T19:23:27.631 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.116084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102282.log 2023-10-18T19:23:27.631 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116084.log.gz 2023-10-18T19:23:27.631 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94313.log 2023-10-18T19:23:27.631 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125228.log.gz 2023-10-18T19:23:27.632 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100375.log 2023-10-18T19:23:27.632 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102282.log.gz 2023-10-18T19:23:27.632 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111804.log 2023-10-18T19:23:27.633 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94313.log.gz 2023-10-18T19:23:27.633 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118415.log 2023-10-18T19:23:27.633 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100375.log.gz 2023-10-18T19:23:27.633 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93499.log 2023-10-18T19:23:27.634 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111804.log.gz 2023-10-18T19:23:27.634 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105139.log 2023-10-18T19:23:27.634 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118415.log.gz 2023-10-18T19:23:27.635 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99295.log 2023-10-18T19:23:27.635 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93499.log.gz 2023-10-18T19:23:27.635 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111933.log 2023-10-18T19:23:27.635 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105139.log.gz 2023-10-18T19:23:27.636 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.99295.log: -- /var/log/ceph/ceph-client.admin.94442.log 2023-10-18T19:23:27.636 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99295.log.gz 2023-10-18T19:23:27.636 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101925.log 2023-10-18T19:23:27.637 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111933.log.gz 2023-10-18T19:23:27.637 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120343.log 2023-10-18T19:23:27.637 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94442.log.gz 2023-10-18T19:23:27.637 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109556.log 2023-10-18T19:23:27.638 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101925.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101925.log.gz 2023-10-18T19:23:27.638 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124098.log 2023-10-18T19:23:27.638 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120343.log.gz 2023-10-18T19:23:27.639 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123498.log 2023-10-18T19:23:27.639 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109556.log.gz 2023-10-18T19:23:27.639 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125594.log 2023-10-18T19:23:27.640 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124098.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124098.log.gz 2023-10-18T19:23:27.640 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93026.log 2023-10-18T19:23:27.640 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123498.log.gz 2023-10-18T19:23:27.641 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126222.log 2023-10-18T19:23:27.641 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125594.log.gz 2023-10-18T19:23:27.641 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110265.log 2023-10-18T19:23:27.641 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93026.log.gz 2023-10-18T19:23:27.642 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95984.log 2023-10-18T19:23:27.642 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126222.log.gz 2023-10-18T19:23:27.642 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129477.log 2023-10-18T19:23:27.643 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110265.log.gz 2023-10-18T19:23:27.643 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128230.log 2023-10-18T19:23:27.643 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95984.log: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.95984.log.gz 2023-10-18T19:23:27.643 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123305.log 2023-10-18T19:23:27.644 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129477.log.gz 2023-10-18T19:23:27.644 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137982.log 2023-10-18T19:23:27.644 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128230.log.gz 2023-10-18T19:23:27.644 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116041.log 2023-10-18T19:23:27.645 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123305.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123305.log.gz 2023-10-18T19:23:27.645 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.137982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133495.log 2023-10-18T19:23:27.645 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.137982.log.gz 2023-10-18T19:23:27.646 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.116041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135370.log 2023-10-18T19:23:27.646 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116041.log.gz 2023-10-18T19:23:27.646 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92304.log 2023-10-18T19:23:27.646 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133495.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133495.log.gz 2023-10-18T19:23:27.647 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100956.log 2023-10-18T19:23:27.647 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135370.log.gz 2023-10-18T19:23:27.647 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113300.log 2023-10-18T19:23:27.647 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.92304.log.gz 2023-10-18T19:23:27.648 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124021.log 2023-10-18T19:23:27.648 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100956.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100956.log.gz 2023-10-18T19:23:27.648 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129554.log 2023-10-18T19:23:27.649 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113300.log.gz 2023-10-18T19:23:27.649 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110527.log 2023-10-18T19:23:27.649 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124021.log.gz 2023-10-18T19:23:27.649 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 /var/log/ceph/ceph-client.admin.129554.log: --verbose -- /var/log/ceph/ceph-client.admin.122854.log 2023-10-18T19:23:27.650 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129554.log.gz 2023-10-18T19:23:27.650 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138945.log 2023-10-18T19:23:27.650 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110527.log.gz 2023-10-18T19:23:27.651 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106031.log 2023-10-18T19:23:27.651 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122854.log.gz 2023-10-18T19:23:27.651 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112673.log 2023-10-18T19:23:27.651 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.138945.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.138945.log.gz 2023-10-18T19:23:27.652 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123348.log 2023-10-18T19:23:27.652 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106031.log.gz 2023-10-18T19:23:27.652 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114528.log 2023-10-18T19:23:27.653 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112673.log.gz 2023-10-18T19:23:27.653 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121005.log 2023-10-18T19:23:27.653 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123348.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123348.log.gz 2023-10-18T19:23:27.653 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123867.log 2023-10-18T19:23:27.654 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114528.log.gz 2023-10-18T19:23:27.654 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118699.log 2023-10-18T19:23:27.654 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121005.log.gz 2023-10-18T19:23:27.655 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97314.log 2023-10-18T19:23:27.655 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123867.log.gz 2023-10-18T19:23:27.655 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105555.log 2023-10-18T19:23:27.655 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118699.log.gz 2023-10-18T19:23:27.656 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111458.log 2023-10-18T19:23:27.656 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97314.log.gz 2023-10-18T19:23:27.656 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95065.log 2023-10-18T19:23:27.656 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105555.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105555.log.gz 2023-10-18T19:23:27.657 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97280.log 2023-10-18T19:23:27.657 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111458.log.gz 2023-10-18T19:23:27.657 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122340.log 2023-10-18T19:23:27.658 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95065.log.gz 2023-10-18T19:23:27.658 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115848.log 2023-10-18T19:23:27.658 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97280.log.gz 2023-10-18T19:23:27.658 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90780.log 2023-10-18T19:23:27.659 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122340.log.gz 2023-10-18T19:23:27.659 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107691.log 2023-10-18T19:23:27.659 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115848.log.gz 2023-10-18T19:23:27.659 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.90780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131040.log 2023-10-18T19:23:27.660 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90780.log.gz 2023-10-18T19:23:27.660 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117549.log 2023-10-18T19:23:27.661 INFO:teuthology.orchestra.run.smithi082.stderr: 56.1% -- replaced with /var/log/ceph/ceph-client.admin.107691.log.gz 2023-10-18T19:23:27.661 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133593.log 2023-10-18T19:23:27.661 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131040.log.gz 2023-10-18T19:23:27.661 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.117549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113797.log 2023-10-18T19:23:27.662 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117549.log.gz 2023-10-18T19:23:27.662 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112234.log 2023-10-18T19:23:27.662 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133593.log.gz 2023-10-18T19:23:27.663 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101075.log 2023-10-18T19:23:27.663 INFO:teuthology.orchestra.run.smithi082.stderr: 52.3% -- replaced with /var/log/ceph/ceph-client.admin.113797.log.gz 2023-10-18T19:23:27.663 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96764.log 2023-10-18T19:23:27.663 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112234.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112234.log.gz 2023-10-18T19:23:27.664 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126899.log 2023-10-18T19:23:27.664 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101075.log.gz 2023-10-18T19:23:27.664 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107272.log 2023-10-18T19:23:27.664 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96764.log.gz 2023-10-18T19:23:27.665 INFO:teuthology.orchestra.run.smithi082.stderr:gzip /var/log/ceph/ceph-client.admin.126899.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.118395.log 2023-10-18T19:23:27.665 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126899.log.gz 2023-10-18T19:23:27.665 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111227.log 2023-10-18T19:23:27.666 INFO:teuthology.orchestra.run.smithi082.stderr: 9.3% -- replaced with /var/log/ceph/ceph-client.admin.107272.log.gz 2023-10-18T19:23:27.666 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124927.log 2023-10-18T19:23:27.666 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118395.log.gz 2023-10-18T19:23:27.667 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126476.log /var/log/ceph/ceph-client.admin.111227.log: 2023-10-18T19:23:27.667 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111227.log.gz 2023-10-18T19:23:27.667 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114140.log 2023-10-18T19:23:27.667 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124927.log.gz 2023-10-18T19:23:27.668 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127170.log 2023-10-18T19:23:27.668 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126476.log.gz 2023-10-18T19:23:27.668 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129203.log 2023-10-18T19:23:27.668 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114140.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114140.log.gz 2023-10-18T19:23:27.669 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113404.log 2023-10-18T19:23:27.669 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127170.log.gz 2023-10-18T19:23:27.669 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111384.log 2023-10-18T19:23:27.670 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129203.log.gz 2023-10-18T19:23:27.670 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119504.log 2023-10-18T19:23:27.670 INFO:teuthology.orchestra.run.smithi082.stderr: 23.2% -- replaced with /var/log/ceph/ceph-client.admin.113404.log.gz 2023-10-18T19:23:27.671 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95194.log 2023-10-18T19:23:27.671 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111384.log.gz 2023-10-18T19:23:27.671 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97620.log 2023-10-18T19:23:27.671 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119504.log.gz 2023-10-18T19:23:27.672 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97603.log 2023-10-18T19:23:27.672 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95194.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95194.log.gz 2023-10-18T19:23:27.672 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121810.log 2023-10-18T19:23:27.673 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97620.log.gz 2023-10-18T19:23:27.673 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91592.log 2023-10-18T19:23:27.673 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97603.log.gz 2023-10-18T19:23:27.673 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 /var/log/ceph/ceph-client.admin.121810.log: --verbose -- /var/log/ceph/ceph-client.admin.100535.log 2023-10-18T19:23:27.674 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121810.log.gz 2023-10-18T19:23:27.674 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110807.log 2023-10-18T19:23:27.674 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91592.log.gz 2023-10-18T19:23:27.675 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138805.log 2023-10-18T19:23:27.675 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100535.log.gz 2023-10-18T19:23:27.675 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105765.log 2023-10-18T19:23:27.675 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110807.log.gz 2023-10-18T19:23:27.676 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.138805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103275.log 2023-10-18T19:23:27.676 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.138805.log.gz 2023-10-18T19:23:27.676 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129950.log 2023-10-18T19:23:27.676 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105765.log.gz 2023-10-18T19:23:27.677 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117915.log 2023-10-18T19:23:27.677 INFO:teuthology.orchestra.run.smithi082.stderr: 23.6% -- replaced with /var/log/ceph/ceph-client.admin.103275.log.gz 2023-10-18T19:23:27.677 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130409.log 2023-10-18T19:23:27.678 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129950.log.gz 2023-10-18T19:23:27.678 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.117915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142084.log 2023-10-18T19:23:27.678 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117915.log.gz 2023-10-18T19:23:27.679 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105010.log 2023-10-18T19:23:27.679 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130409.log.gz 2023-10-18T19:23:27.679 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.142084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101772.log 2023-10-18T19:23:27.679 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.142084.log.gz 2023-10-18T19:23:27.680 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113862.log 2023-10-18T19:23:27.680 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105010.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105010.log.gz 2023-10-18T19:23:27.680 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121896.log 2023-10-18T19:23:27.680 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101772.log.gz 2023-10-18T19:23:27.681 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101466.log 2023-10-18T19:23:27.681 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113862.log.gz 2023-10-18T19:23:27.682 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.121896.log: -- /var/log/ceph/ceph-client.admin.118237.log 2023-10-18T19:23:27.682 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121896.log.gz 2023-10-18T19:23:27.682 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123923.log 2023-10-18T19:23:27.682 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101466.log.gz 2023-10-18T19:23:27.683 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103779.log 2023-10-18T19:23:27.683 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118237.log.gz 2023-10-18T19:23:27.683 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133060.log 2023-10-18T19:23:27.684 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123923.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123923.log.gz 2023-10-18T19:23:27.684 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102855.log 2023-10-18T19:23:27.684 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103779.log.gz 2023-10-18T19:23:27.684 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130602.log 2023-10-18T19:23:27.685 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133060.log.gz 2023-10-18T19:23:27.685 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133812.log 2023-10-18T19:23:27.685 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102855.log: 23.6% -- replaced with /var/log/ceph/ceph-client.admin.102855.log.gz 2023-10-18T19:23:27.685 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93810.log 2023-10-18T19:23:27.686 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130602.log.gz 2023-10-18T19:23:27.686 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90064.log 2023-10-18T19:23:27.686 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133812.log.gz 2023-10-18T19:23:27.686 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135456.log 2023-10-18T19:23:27.687 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93810.log: 6.4% -- replaced with /var/log/ceph/ceph-client.admin.93810.log.gz 2023-10-18T19:23:27.687 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.90064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127724.log 2023-10-18T19:23:27.687 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90064.log.gz 2023-10-18T19:23:27.688 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120041.log 2023-10-18T19:23:27.688 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135456.log.gz 2023-10-18T19:23:27.688 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99995.log 2023-10-18T19:23:27.688 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127724.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127724.log.gz 2023-10-18T19:23:27.689 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114162.log 2023-10-18T19:23:27.689 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120041.log.gz 2023-10-18T19:23:27.689 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122617.log 2023-10-18T19:23:27.689 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99995.log.gz 2023-10-18T19:23:27.690 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108491.log 2023-10-18T19:23:27.690 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114162.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114162.log.gz 2023-10-18T19:23:27.690 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109535.log 2023-10-18T19:23:27.691 INFO:teuthology.orchestra.run.smithi082.stderr: 22.5% -- replaced with /var/log/ceph/ceph-client.admin.122617.log.gz 2023-10-18T19:23:27.691 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127805.log 2023-10-18T19:23:27.691 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108491.log.gz 2023-10-18T19:23:27.692 INFO:teuthology.orchestra.run.smithi082.stderr:gzip /var/log/ceph/ceph-client.admin.109535.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.129243.log 2023-10-18T19:23:27.692 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109535.log.gz 2023-10-18T19:23:27.692 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93004.log 2023-10-18T19:23:27.693 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127805.log.gz 2023-10-18T19:23:27.693 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122983.log 2023-10-18T19:23:27.693 INFO:teuthology.orchestra.run.smithi082.stderr: 56.3% -- replaced with /var/log/ceph/ceph-client.admin.129243.log.gz 2023-10-18T19:23:27.694 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115274.log /var/log/ceph/ceph-client.admin.93004.log: 2023-10-18T19:23:27.694 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93004.log.gz 2023-10-18T19:23:27.694 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125164.log 2023-10-18T19:23:27.694 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122983.log.gz 2023-10-18T19:23:27.695 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108210.log 2023-10-18T19:23:27.695 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115274.log.gz 2023-10-18T19:23:27.695 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113543.log 2023-10-18T19:23:27.696 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125164.log.gz 2023-10-18T19:23:27.696 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114507.log 2023-10-18T19:23:27.696 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108210.log.gz 2023-10-18T19:23:27.696 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128928.log 2023-10-18T19:23:27.697 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113543.log.gz 2023-10-18T19:23:27.697 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126131.log 2023-10-18T19:23:27.697 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114507.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114507.log.gz 2023-10-18T19:23:27.698 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107524.log 2023-10-18T19:23:27.698 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128928.log.gz 2023-10-18T19:23:27.698 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113425.log 2023-10-18T19:23:27.698 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126131.log.gz 2023-10-18T19:23:27.699 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 /var/log/ceph/ceph-client.admin.107524.log: --verbose -- /var/log/ceph/ceph-client.admin.91442.log 2023-10-18T19:23:27.699 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107524.log.gz 2023-10-18T19:23:27.699 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101109.log 2023-10-18T19:23:27.699 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113425.log.gz 2023-10-18T19:23:27.700 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97246.log 2023-10-18T19:23:27.700 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91442.log.gz 2023-10-18T19:23:27.700 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.101109.log: -- /var/log/ceph/ceph-client.admin.132786.log 2023-10-18T19:23:27.701 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101109.log.gz 2023-10-18T19:23:27.701 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121444.log 2023-10-18T19:23:27.701 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97246.log.gz 2023-10-18T19:23:27.701 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125486.log 2023-10-18T19:23:27.702 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132786.log.gz 2023-10-18T19:23:27.702 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127581.log 2023-10-18T19:23:27.703 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121444.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121444.log.gz 2023-10-18T19:23:27.703 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109470.log 2023-10-18T19:23:27.703 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125486.log.gz 2023-10-18T19:23:27.703 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115151.log 2023-10-18T19:23:27.704 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127581.log.gz 2023-10-18T19:23:27.704 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109470.log: /var/log/ceph/ceph-client.admin.141740.log 2023-10-18T19:23:27.704 INFO:teuthology.orchestra.run.smithi082.stderr: 22.4% -- replaced with /var/log/ceph/ceph-client.admin.109470.log.gz 2023-10-18T19:23:27.704 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100854.log 2023-10-18T19:23:27.705 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.115151.log.gz 2023-10-18T19:23:27.705 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.141740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98504.log 2023-10-18T19:23:27.705 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.141740.log.gz 2023-10-18T19:23:27.705 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119093.log 2023-10-18T19:23:27.706 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100854.log.gz 2023-10-18T19:23:27.706 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113819.log 2023-10-18T19:23:27.706 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.98504.log.gz 2023-10-18T19:23:27.707 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100684.log 2023-10-18T19:23:27.707 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119093.log.gz 2023-10-18T19:23:27.707 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 /var/log/ceph/ceph-client.admin.113819.log: --verbose -- /var/log/ceph/ceph-client.admin.105959.log 2023-10-18T19:23:27.708 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113819.log.gz 2023-10-18T19:23:27.708 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133855.log 2023-10-18T19:23:27.708 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100684.log.gz 2023-10-18T19:23:27.708 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104220.log 2023-10-18T19:23:27.709 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105959.log.gz 2023-10-18T19:23:27.709 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120236.log 2023-10-18T19:23:27.709 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133855.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133855.log.gz 2023-10-18T19:23:27.710 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122498.log 2023-10-18T19:23:27.710 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104220.log.gz 2023-10-18T19:23:27.710 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118595.log 2023-10-18T19:23:27.711 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120236.log.gz 2023-10-18T19:23:27.711 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111615.log 2023-10-18T19:23:27.711 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122498.log.gz 2023-10-18T19:23:27.711 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115719.log 2023-10-18T19:23:27.712 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118595.log.gz 2023-10-18T19:23:27.712 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121874.log 2023-10-18T19:23:27.712 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111615.log.gz 2023-10-18T19:23:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.82611.log 2023-10-18T19:23:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115719.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115719.log.gz 2023-10-18T19:23:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103338.log 2023-10-18T19:23:27.713 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121874.log.gz 2023-10-18T19:23:27.714 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph.tmp-client.admin.82611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93831.log 2023-10-18T19:23:27.714 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.82611.log.gz 2023-10-18T19:23:27.714 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99495.log /var/log/ceph/ceph-client.admin.103338.log: 2023-10-18T19:23:27.715 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103338.log.gz 2023-10-18T19:23:27.715 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92221.log 2023-10-18T19:23:27.715 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93831.log.gz 2023-10-18T19:23:27.715 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92247.log 2023-10-18T19:23:27.716 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99495.log.gz 2023-10-18T19:23:27.716 INFO:teuthology.orchestra.run.smithi082.stderr:gzip /var/log/ceph/ceph-client.admin.92221.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.93284.log 2023-10-18T19:23:27.716 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92221.log.gz 2023-10-18T19:23:27.717 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108528.log 2023-10-18T19:23:27.717 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92247.log.gz 2023-10-18T19:23:27.717 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130898.log 2023-10-18T19:23:27.717 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93284.log.gz 2023-10-18T19:23:27.718 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127367.log 2023-10-18T19:23:27.718 INFO:teuthology.orchestra.run.smithi082.stderr: 31.8% -- replaced with /var/log/ceph/ceph-client.admin.108528.log.gz 2023-10-18T19:23:27.718 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104473.log 2023-10-18T19:23:27.719 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130898.log.gz 2023-10-18T19:23:27.719 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100355.log 2023-10-18T19:23:27.719 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127367.log.gz 2023-10-18T19:23:27.719 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128270.log 2023-10-18T19:23:27.720 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104473.log.gz 2023-10-18T19:23:27.720 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98232.log 2023-10-18T19:23:27.720 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100355.log.gz 2023-10-18T19:23:27.720 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114657.log 2023-10-18T19:23:27.721 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128270.log.gz 2023-10-18T19:23:27.721 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125379.log 2023-10-18T19:23:27.721 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98232.log: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.98232.log.gz 2023-10-18T19:23:27.722 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112693.log 2023-10-18T19:23:27.722 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114657.log.gz 2023-10-18T19:23:27.722 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120106.log 2023-10-18T19:23:27.722 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125379.log.gz 2023-10-18T19:23:27.723 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133941.log 2023-10-18T19:23:27.723 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112693.log.gz 2023-10-18T19:23:27.723 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110893.log 2023-10-18T19:23:27.724 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120106.log.gz 2023-10-18T19:23:27.724 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103821.log 2023-10-18T19:23:27.724 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133941.log.gz 2023-10-18T19:23:27.725 INFO:teuthology.orchestra.run.smithi082.stderr:gzip /var/log/ceph/ceph-client.admin.110893.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.128823.log 2023-10-18T19:23:27.725 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110893.log.gz 2023-10-18T19:23:27.725 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120063.log 2023-10-18T19:23:27.726 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103821.log.gz 2023-10-18T19:23:27.726 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91313.log 2023-10-18T19:23:27.726 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128823.log.gz 2023-10-18T19:23:27.726 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98130.log 2023-10-18T19:23:27.727 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120063.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120063.log.gz 2023-10-18T19:23:27.727 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123477.log 2023-10-18T19:23:27.727 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91313.log.gz 2023-10-18T19:23:27.728 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97756.log 2023-10-18T19:23:27.728 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.98130.log.gz 2023-10-18T19:23:27.728 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104073.log 2023-10-18T19:23:27.728 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123477.log.gz 2023-10-18T19:23:27.729 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111560.log 2023-10-18T19:23:27.729 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97756.log.gz 2023-10-18T19:23:27.729 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109407.log 2023-10-18T19:23:27.730 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104073.log.gz 2023-10-18T19:23:27.730 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111955.log 2023-10-18T19:23:27.731 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111560.log.gz 2023-10-18T19:23:27.731 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95828.log 2023-10-18T19:23:27.731 INFO:teuthology.orchestra.run.smithi082.stderr: 23.2% -- replaced with /var/log/ceph/ceph-client.admin.109407.log.gz 2023-10-18T19:23:27.732 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99115.log 2023-10-18T19:23:27.732 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111955.log.gz 2023-10-18T19:23:27.733 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120193.log 2023-10-18T19:23:27.733 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95828.log.gz 2023-10-18T19:23:27.734 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127940.log 2023-10-18T19:23:27.734 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99115.log.gz 2023-10-18T19:23:27.735 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120279.log 2023-10-18T19:23:27.735 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120193.log.gz 2023-10-18T19:23:27.736 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133308.log 2023-10-18T19:23:27.736 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127940.log.gz 2023-10-18T19:23:27.737 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110137.log 2023-10-18T19:23:27.737 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120279.log.gz 2023-10-18T19:23:27.738 INFO:teuthology.orchestra.run.smithi082.stderr:gzip /var/log/ceph/ceph-client.admin.133308.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.97569.log 2023-10-18T19:23:27.738 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133308.log.gz 2023-10-18T19:23:27.739 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120408.log 2023-10-18T19:23:27.739 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110137.log.gz 2023-10-18T19:23:27.740 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111912.log 2023-10-18T19:23:27.740 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.97569.log.gz 2023-10-18T19:23:27.740 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105514.log 2023-10-18T19:23:27.741 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120408.log.gz 2023-10-18T19:23:27.741 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97824.log 2023-10-18T19:23:27.741 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111912.log.gz 2023-10-18T19:23:27.742 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101908.log 2023-10-18T19:23:27.742 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105514.log.gz 2023-10-18T19:23:27.742 INFO:teuthology.orchestra.run.smithi082.stderr:gzip /var/log/ceph/ceph-client.admin.97824.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.127899.log 2023-10-18T19:23:27.742 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97824.log.gz 2023-10-18T19:23:27.743 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112843.log 2023-10-18T19:23:27.743 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101908.log.gz 2023-10-18T19:23:27.743 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127899.log: 50.9% -- replaced with /var/log/ceph/ceph-client.admin.127899.log.gz 2023-10-18T19:23:27.743 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90274.log 2023-10-18T19:23:27.744 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112843.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112843.log.gz 2023-10-18T19:23:27.744 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129594.log 2023-10-18T19:23:27.744 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.90274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134479.log 2023-10-18T19:23:27.745 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90274.log.gz 2023-10-18T19:23:27.745 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130430.log 2023-10-18T19:23:27.745 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129594.log.gz 2023-10-18T19:23:27.745 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118637.log 2023-10-18T19:23:27.746 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134479.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134479.log.gz 2023-10-18T19:23:27.746 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115504.log 2023-10-18T19:23:27.746 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130430.log.gz 2023-10-18T19:23:27.747 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102665.log 2023-10-18T19:23:27.747 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118637.log.gz 2023-10-18T19:23:27.747 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124301.log 2023-10-18T19:23:27.747 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115504.log.gz 2023-10-18T19:23:27.748 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114183.log 2023-10-18T19:23:27.748 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102665.log.gz 2023-10-18T19:23:27.748 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139809.log 2023-10-18T19:23:27.748 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124301.log.gz 2023-10-18T19:23:27.749 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138074.log 2023-10-18T19:23:27.749 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114183.log.gz 2023-10-18T19:23:27.749 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.139809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107191.log 2023-10-18T19:23:27.750 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.139809.log.gz 2023-10-18T19:23:27.750 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.138074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111654.log 2023-10-18T19:23:27.750 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.138074.log.gz 2023-10-18T19:23:27.750 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94871.log 2023-10-18T19:23:27.751 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107191.log.gz 2023-10-18T19:23:27.751 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109685.log 2023-10-18T19:23:27.751 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111654.log.gz 2023-10-18T19:23:27.752 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93133.log 2023-10-18T19:23:27.752 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94871.log.gz 2023-10-18T19:23:27.752 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117764.log 2023-10-18T19:23:27.752 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109685.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109685.log.gz 2023-10-18T19:23:27.753 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95473.log 2023-10-18T19:23:27.753 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93133.log.gz 2023-10-18T19:23:27.753 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.117764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114636.log 2023-10-18T19:23:27.754 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117764.log.gz 2023-10-18T19:23:27.754 INFO:teuthology.orchestra.run.smithi082.stderr:gzip /var/log/ceph/ceph-client.admin.95473.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.110914.log 2023-10-18T19:23:27.754 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95473.log.gz 2023-10-18T19:23:27.754 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99155.log 2023-10-18T19:23:27.755 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114636.log.gz 2023-10-18T19:23:27.755 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124341.log 2023-10-18T19:23:27.755 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110914.log.gz 2023-10-18T19:23:27.755 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104903.log 2023-10-18T19:23:27.756 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99155.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99155.log.gz 2023-10-18T19:23:27.756 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126667.log 2023-10-18T19:23:27.756 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124341.log.gz 2023-10-18T19:23:27.757 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138848.log 2023-10-18T19:23:27.757 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104903.log.gz 2023-10-18T19:23:27.757 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107154.log 2023-10-18T19:23:27.757 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126667.log.gz 2023-10-18T19:23:27.758 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.138848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122400.log 2023-10-18T19:23:27.758 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.138848.log.gz 2023-10-18T19:23:27.758 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96292.log 2023-10-18T19:23:27.759 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.107154.log.gz 2023-10-18T19:23:27.759 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106730.log 2023-10-18T19:23:27.759 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122400.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122400.log.gz 2023-10-18T19:23:27.759 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119175.log 2023-10-18T19:23:27.760 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96292.log.gz 2023-10-18T19:23:27.760 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131348.log 2023-10-18T19:23:27.760 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106730.log.gz 2023-10-18T19:23:27.761 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132743.log 2023-10-18T19:23:27.761 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119175.log.gz 2023-10-18T19:23:27.761 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127311.log 2023-10-18T19:23:27.761 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131348.log.gz 2023-10-18T19:23:27.762 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141144.log 2023-10-18T19:23:27.762 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132743.log.gz 2023-10-18T19:23:27.762 INFO:teuthology.orchestra.run.smithi082.stderr:gzip /var/log/ceph/ceph-client.admin.127311.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.138544.log 2023-10-18T19:23:27.762 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127311.log.gz 2023-10-18T19:23:27.763 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.141144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95883.log 2023-10-18T19:23:27.763 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.141144.log.gz 2023-10-18T19:23:27.763 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.138544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118822.log 2023-10-18T19:23:27.764 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.138544.log.gz 2023-10-18T19:23:27.764 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97501.log 2023-10-18T19:23:27.764 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95883.log.gz 2023-10-18T19:23:27.765 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111174.log 2023-10-18T19:23:27.765 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118822.log.gz 2023-10-18T19:23:27.765 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130978.log 2023-10-18T19:23:27.765 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97501.log.gz 2023-10-18T19:23:27.766 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129400.log 2023-10-18T19:23:27.766 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111174.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111174.log.gz 2023-10-18T19:23:27.766 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132614.log 2023-10-18T19:23:27.766 INFO:teuthology.orchestra.run.smithi082.stderr: 20.6% -- replaced with /var/log/ceph/ceph-client.admin.130978.log.gz 2023-10-18T19:23:27.767 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131947.log 2023-10-18T19:23:27.767 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129400.log.gz 2023-10-18T19:23:27.767 INFO:teuthology.orchestra.run.smithi082.stderr:gzip /var/log/ceph/ceph-client.admin.132614.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.106689.log 2023-10-18T19:23:27.768 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132614.log.gz 2023-10-18T19:23:27.768 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121960.log 2023-10-18T19:23:27.768 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131947.log.gz 2023-10-18T19:23:27.768 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100315.log 2023-10-18T19:23:27.769 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106689.log.gz 2023-10-18T19:23:27.769 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99435.log 2023-10-18T19:23:27.769 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121960.log.gz 2023-10-18T19:23:27.769 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102316.log 2023-10-18T19:23:27.770 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100315.log.gz 2023-10-18T19:23:27.770 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121509.log 2023-10-18T19:23:27.770 INFO:teuthology.orchestra.run.smithi082.stderr: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.99435.log.gz 2023-10-18T19:23:27.770 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111087.log 2023-10-18T19:23:27.771 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102316.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102316.log.gz 2023-10-18T19:23:27.771 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109186.log 2023-10-18T19:23:27.771 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121509.log.gz 2023-10-18T19:23:27.772 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95675.log 2023-10-18T19:23:27.772 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111087.log.gz 2023-10-18T19:23:27.772 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139675.log 2023-10-18T19:23:27.772 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109186.log.gz 2023-10-18T19:23:27.773 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118498.log 2023-10-18T19:23:27.773 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95675.log.gz 2023-10-18T19:23:27.773 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.139675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123888.log 2023-10-18T19:23:27.774 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.139675.log.gz 2023-10-18T19:23:27.774 INFO:teuthology.orchestra.run.smithi082.stderr:gzip /var/log/ceph/ceph-client.admin.118498.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.111633.log 2023-10-18T19:23:27.774 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118498.log.gz 2023-10-18T19:23:27.775 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98351.log 2023-10-18T19:23:27.775 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123888.log.gz 2023-10-18T19:23:27.775 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103674.log 2023-10-18T19:23:27.775 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111633.log.gz 2023-10-18T19:23:27.776 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92962.log 2023-10-18T19:23:27.776 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.98351.log.gz 2023-10-18T19:23:27.776 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90906.log 2023-10-18T19:23:27.777 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103674.log.gz 2023-10-18T19:23:27.777 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109879.log 2023-10-18T19:23:27.777 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92962.log.gz 2023-10-18T19:23:27.777 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.90906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111138.log 2023-10-18T19:23:27.778 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90906.log.gz 2023-10-18T19:23:27.778 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130817.log 2023-10-18T19:23:27.778 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109879.log.gz 2023-10-18T19:23:27.779 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107252.log 2023-10-18T19:23:27.779 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111138.log.gz 2023-10-18T19:23:27.779 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130817.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-18T19:23:27.779 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130817.log.gz 2023-10-18T19:23:27.780 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127958.log 2023-10-18T19:23:27.780 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107252.log.gz 2023-10-18T19:23:27.780 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100973.log 2023-10-18T19:23:27.780 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140229.log 2023-10-18T19:23:27.781 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127958.log.gz 2023-10-18T19:23:27.781 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112277.log 2023-10-18T19:23:27.781 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100973.log.gz 2023-10-18T19:23:27.782 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.140229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90022.log 2023-10-18T19:23:27.782 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.140229.log.gz 2023-10-18T19:23:27.782 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139239.log 2023-10-18T19:23:27.782 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112277.log.gz 2023-10-18T19:23:27.783 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.90022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140413.log 2023-10-18T19:23:27.783 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90022.log.gz 2023-10-18T19:23:27.783 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.139239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126591.log 2023-10-18T19:23:27.784 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.139239.log.gz 2023-10-18T19:23:27.784 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.140413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130215.log 2023-10-18T19:23:27.784 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.140413.log.gz 2023-10-18T19:23:27.784 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125615.log 2023-10-18T19:23:27.785 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126591.log.gz 2023-10-18T19:23:27.785 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128766.log 2023-10-18T19:23:27.785 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130215.log.gz 2023-10-18T19:23:27.785 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120773.log 2023-10-18T19:23:27.786 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125615.log.gz 2023-10-18T19:23:27.786 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92266.log 2023-10-18T19:23:27.786 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128766.log.gz 2023-10-18T19:23:27.786 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96206.log 2023-10-18T19:23:27.787 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120773.log.gz 2023-10-18T19:23:27.787 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102443.log 2023-10-18T19:23:27.787 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.92266.log.gz 2023-10-18T19:23:27.788 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94721.log 2023-10-18T19:23:27.788 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96206.log.gz 2023-10-18T19:23:27.788 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129008.log 2023-10-18T19:23:27.788 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102443.log.gz 2023-10-18T19:23:27.789 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115547.log 2023-10-18T19:23:27.789 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94721.log.gz 2023-10-18T19:23:27.789 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124669.log 2023-10-18T19:23:27.789 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129008.log.gz 2023-10-18T19:23:27.790 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109857.log 2023-10-18T19:23:27.790 INFO:teuthology.orchestra.run.smithi082.stderr: 26.8% -- replaced with /var/log/ceph/ceph-client.admin.115547.log.gz 2023-10-18T19:23:27.790 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142526.log 2023-10-18T19:23:27.791 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124669.log.gz 2023-10-18T19:23:27.791 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103716.log 2023-10-18T19:23:27.791 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109857.log.gz 2023-10-18T19:23:27.791 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.142526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120601.log 2023-10-18T19:23:27.792 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.142526.log.gz 2023-10-18T19:23:27.792 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124884.log 2023-10-18T19:23:27.792 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103716.log.gz 2023-10-18T19:23:27.792 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112019.log 2023-10-18T19:23:27.793 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120601.log.gz 2023-10-18T19:23:27.793 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105808.log 2023-10-18T19:23:27.793 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124884.log.gz 2023-10-18T19:23:27.794 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132356.log 2023-10-18T19:23:27.794 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112019.log.gz 2023-10-18T19:23:27.794 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123370.log 2023-10-18T19:23:27.795 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105808.log.gz 2023-10-18T19:23:27.795 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87080.log 2023-10-18T19:23:27.795 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132356.log.gz 2023-10-18T19:23:27.795 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141790.log 2023-10-18T19:23:27.796 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123370.log.gz 2023-10-18T19:23:27.796 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.87080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132979.log 2023-10-18T19:23:27.796 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87080.log.gz 2023-10-18T19:23:27.796 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.141790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94292.log 2023-10-18T19:23:27.797 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.141790.log.gz 2023-10-18T19:23:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129911.log 2023-10-18T19:23:27.797 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132979.log.gz 2023-10-18T19:23:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111869.log 2023-10-18T19:23:27.798 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94292.log.gz 2023-10-18T19:23:27.798 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132829.log 2023-10-18T19:23:27.798 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129911.log.gz 2023-10-18T19:23:27.799 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123766.log 2023-10-18T19:23:27.799 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111869.log.gz 2023-10-18T19:23:27.799 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108429.log 2023-10-18T19:23:27.799 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132829.log.gz 2023-10-18T19:23:27.800 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114464.log 2023-10-18T19:23:27.800 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123766.log.gz 2023-10-18T19:23:27.800 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108569.log 2023-10-18T19:23:27.801 INFO:teuthology.orchestra.run.smithi082.stderr: 51.1% -- replaced with /var/log/ceph/ceph-client.admin.108429.log.gz 2023-10-18T19:23:27.801 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138334.log 2023-10-18T19:23:27.801 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114464.log.gz 2023-10-18T19:23:27.802 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128968.log 2023-10-18T19:23:27.802 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108569.log.gz 2023-10-18T19:23:27.802 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.138334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104817.log 2023-10-18T19:23:27.803 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.138334.log.gz 2023-10-18T19:23:27.803 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113754.log 2023-10-18T19:23:27.803 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128968.log.gz 2023-10-18T19:23:27.803 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103086.log 2023-10-18T19:23:27.804 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104817.log.gz 2023-10-18T19:23:27.804 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139113.log 2023-10-18T19:23:27.804 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113754.log.gz 2023-10-18T19:23:27.805 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133750.log 2023-10-18T19:23:27.805 INFO:teuthology.orchestra.run.smithi082.stderr: 22.5% -- replaced with /var/log/ceph/ceph-client.admin.103086.log.gz 2023-10-18T19:23:27.805 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.139113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122218.log 2023-10-18T19:23:27.805 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.139113.log.gz 2023-10-18T19:23:27.806 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108022.log 2023-10-18T19:23:27.806 INFO:teuthology.orchestra.run.smithi082.stderr: 55.7% -- replaced with /var/log/ceph/ceph-client.admin.133750.log.gz 2023-10-18T19:23:27.807 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92239.log 2023-10-18T19:23:27.807 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122218.log.gz 2023-10-18T19:23:27.808 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101976.log 2023-10-18T19:23:27.808 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108022.log.gz 2023-10-18T19:23:27.809 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118904.log 2023-10-18T19:23:27.809 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92239.log.gz 2023-10-18T19:23:27.810 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131226.log 2023-10-18T19:23:27.810 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101976.log.gz 2023-10-18T19:23:27.811 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134833.log 2023-10-18T19:23:27.811 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118904.log.gz 2023-10-18T19:23:27.811 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95301.log 2023-10-18T19:23:27.812 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131226.log.gz 2023-10-18T19:23:27.812 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120300.log 2023-10-18T19:23:27.813 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134833.log.gz 2023-10-18T19:23:27.813 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127764.log 2023-10-18T19:23:27.813 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95301.log.gz 2023-10-18T19:23:27.813 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126998.log 2023-10-18T19:23:27.814 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120300.log.gz 2023-10-18T19:23:27.814 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108309.log 2023-10-18T19:23:27.814 INFO:teuthology.orchestra.run.smithi082.stderr: 53.1% -- replaced with /var/log/ceph/ceph-client.admin.127764.log.gz 2023-10-18T19:23:27.815 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121298.log 2023-10-18T19:23:27.815 INFO:teuthology.orchestra.run.smithi082.stderr: 56.9% -- replaced with /var/log/ceph/ceph-client.admin.126998.log.gz 2023-10-18T19:23:27.815 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138208.log 2023-10-18T19:23:27.815 INFO:teuthology.orchestra.run.smithi082.stderr: 22.4% -- replaced with /var/log/ceph/ceph-client.admin.108309.log.gz 2023-10-18T19:23:27.816 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121745.log 2023-10-18T19:23:27.816 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121298.log.gz 2023-10-18T19:23:27.816 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.138208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116106.log 2023-10-18T19:23:27.817 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.138208.log.gz 2023-10-18T19:23:27.817 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92628.log 2023-10-18T19:23:27.817 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121745.log.gz 2023-10-18T19:23:27.817 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.116106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108330.log 2023-10-18T19:23:27.818 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116106.log.gz 2023-10-18T19:23:27.818 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128588.log 2023-10-18T19:23:27.818 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92628.log.gz 2023-10-18T19:23:27.819 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117829.log 2023-10-18T19:23:27.819 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108330.log.gz 2023-10-18T19:23:27.819 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111740.log 2023-10-18T19:23:27.819 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128588.log.gz 2023-10-18T19:23:27.820 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.117829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105787.log 2023-10-18T19:23:27.820 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117829.log.gz 2023-10-18T19:23:27.820 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113280.log 2023-10-18T19:23:27.821 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111740.log.gz 2023-10-18T19:23:27.821 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96872.log 2023-10-18T19:23:27.821 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105787.log.gz 2023-10-18T19:23:27.821 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133791.log 2023-10-18T19:23:27.822 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113280.log.gz 2023-10-18T19:23:27.822 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127621.log 2023-10-18T19:23:27.822 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.96872.log.gz 2023-10-18T19:23:27.822 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105053.log 2023-10-18T19:23:27.823 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133791.log.gz 2023-10-18T19:23:27.823 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99955.log 2023-10-18T19:23:27.823 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127621.log.gz 2023-10-18T19:23:27.824 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102939.log 2023-10-18T19:23:27.824 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105053.log.gz 2023-10-18T19:23:27.824 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118925.log 2023-10-18T19:23:27.824 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99955.log.gz 2023-10-18T19:23:27.825 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129775.log 2023-10-18T19:23:27.825 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102939.log.gz 2023-10-18T19:23:27.825 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125959.log 2023-10-18T19:23:27.826 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118925.log.gz 2023-10-18T19:23:27.826 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134769.log 2023-10-18T19:23:27.826 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129775.log.gz 2023-10-18T19:23:27.826 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126418.log 2023-10-18T19:23:27.827 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125959.log.gz 2023-10-18T19:23:27.827 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134769.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134769.log.gz 2023-10-18T19:23:27.827 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141396.log 2023-10-18T19:23:27.828 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126418.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126418.log.gz 2023-10-18T19:23:27.828 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99455.log 2023-10-18T19:23:27.828 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.141396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100555.log 2023-10-18T19:23:27.829 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.141396.log.gz 2023-10-18T19:23:27.829 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122239.log 2023-10-18T19:23:27.829 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99455.log.gz 2023-10-18T19:23:27.829 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121193.log 2023-10-18T19:23:27.830 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100555.log.gz 2023-10-18T19:23:27.830 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140918.log 2023-10-18T19:23:27.830 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122239.log.gz 2023-10-18T19:23:27.831 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121193.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121193.log.gz 2023-10-18T19:23:27.831 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118883.log 2023-10-18T19:23:27.831 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.140918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113156.log 2023-10-18T19:23:27.831 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.140918.log.gz 2023-10-18T19:23:27.832 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106225.log 2023-10-18T19:23:27.832 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118883.log.gz 2023-10-18T19:23:27.832 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130129.log 2023-10-18T19:23:27.833 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113156.log.gz 2023-10-18T19:23:27.833 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108668.log 2023-10-18T19:23:27.833 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106225.log.gz 2023-10-18T19:23:27.833 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92798.log 2023-10-18T19:23:27.834 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130129.log.gz 2023-10-18T19:23:27.834 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121277.log 2023-10-18T19:23:27.834 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108668.log.gz 2023-10-18T19:23:27.835 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135112.log 2023-10-18T19:23:27.835 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92798.log.gz 2023-10-18T19:23:27.835 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134328.log 2023-10-18T19:23:27.835 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121277.log.gz 2023-10-18T19:23:27.836 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115087.log 2023-10-18T19:23:27.836 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135112.log.gz 2023-10-18T19:23:27.836 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134328.log: gzip 0.0%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.134328.log.gz-- 2023-10-18T19:23:27.836 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115001.log 2023-10-18T19:23:27.837 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99095.log 2023-10-18T19:23:27.837 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115087.log.gz 2023-10-18T19:23:27.837 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90568.log 2023-10-18T19:23:27.837 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115001.log.gz 2023-10-18T19:23:27.838 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99095.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99095.log.gz 2023-10-18T19:23:27.838 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132549.log 2023-10-18T19:23:27.838 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.90568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115316.log 2023-10-18T19:23:27.839 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90568.log.gz 2023-10-18T19:23:27.839 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101364.log 2023-10-18T19:23:27.839 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132549.log.gz 2023-10-18T19:23:27.840 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105246.log 2023-10-18T19:23:27.840 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115316.log.gz 2023-10-18T19:23:27.840 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129815.log 2023-10-18T19:23:27.840 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101364.log.gz 2023-10-18T19:23:27.841 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120966.log 2023-10-18T19:23:27.841 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105246.log.gz 2023-10-18T19:23:27.841 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127020.log 2023-10-18T19:23:27.841 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129815.log.gz 2023-10-18T19:23:27.842 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129341.log 2023-10-18T19:23:27.842 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120966.log.gz 2023-10-18T19:23:27.842 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100075.log 2023-10-18T19:23:27.843 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127020.log.gz 2023-10-18T19:23:27.843 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92241.log 2023-10-18T19:23:27.843 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129341.log.gz 2023-10-18T19:23:27.843 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122746.log 2023-10-18T19:23:27.844 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100075.log.gz 2023-10-18T19:23:27.844 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98164.log 2023-10-18T19:23:27.844 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92241.log.gz 2023-10-18T19:23:27.845 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105161.log 2023-10-18T19:23:27.845 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122746.log.gz 2023-10-18T19:23:27.845 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119633.log 2023-10-18T19:23:27.845 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.98164.log.gz 2023-10-18T19:23:27.846 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106182.log 2023-10-18T19:23:27.846 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105161.log.gz 2023-10-18T19:23:27.846 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135306.log 2023-10-18T19:23:27.847 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119633.log.gz 2023-10-18T19:23:27.847 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113446.log 2023-10-18T19:23:27.847 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106182.log.gz 2023-10-18T19:23:27.847 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122154.log 2023-10-18T19:23:27.848 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135306.log.gz 2023-10-18T19:23:27.848 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95693.log 2023-10-18T19:23:27.848 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.113446.log.gz 2023-10-18T19:23:27.848 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97263.log 2023-10-18T19:23:27.849 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122154.log.gz 2023-10-18T19:23:27.849 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111043.log 2023-10-18T19:23:27.849 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95693.log.gz 2023-10-18T19:23:27.849 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98775.log 2023-10-18T19:23:27.850 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97263.log.gz 2023-10-18T19:23:27.850 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117443.log 2023-10-18T19:23:27.850 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111043.log.gz 2023-10-18T19:23:27.851 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113883.log 2023-10-18T19:23:27.851 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98775.log.gz 2023-10-18T19:23:27.851 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.117443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131732.log 2023-10-18T19:23:27.851 INFO:teuthology.orchestra.run.smithi082.stderr: 56.5% -- replaced with /var/log/ceph/ceph-client.admin.117443.log.gz 2023-10-18T19:23:27.852 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124525.log 2023-10-18T19:23:27.852 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113883.log.gz 2023-10-18T19:23:27.852 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103632.log 2023-10-18T19:23:27.853 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131732.log.gz 2023-10-18T19:23:27.853 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94184.log 2023-10-18T19:23:27.853 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124525.log.gz 2023-10-18T19:23:27.853 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99635.log 2023-10-18T19:23:27.854 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103632.log.gz 2023-10-18T19:23:27.854 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120924.log 2023-10-18T19:23:27.854 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94184.log.gz 2023-10-18T19:23:27.855 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98715.log 2023-10-18T19:23:27.855 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99635.log.gz 2023-10-18T19:23:27.855 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93391.log 2023-10-18T19:23:27.855 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120924.log.gz 2023-10-18T19:23:27.856 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130280.log 2023-10-18T19:23:27.856 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98715.log.gz 2023-10-18T19:23:27.856 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109028.log 2023-10-18T19:23:27.856 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93391.log.gz 2023-10-18T19:23:27.857 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102876.log 2023-10-18T19:23:27.857 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130280.log.gz 2023-10-18T19:23:27.857 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130258.log 2023-10-18T19:23:27.857 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109028.log.gz 2023-10-18T19:23:27.858 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120752.log 2023-10-18T19:23:27.858 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102876.log.gz 2023-10-18T19:23:27.858 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112797.log 2023-10-18T19:23:27.859 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130258.log.gz 2023-10-18T19:23:27.859 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98995.log 2023-10-18T19:23:27.859 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120752.log.gz 2023-10-18T19:23:27.860 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128310.log 2023-10-18T19:23:27.860 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112797.log.gz 2023-10-18T19:23:27.860 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95366.log 2023-10-18T19:23:27.860 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98995.log.gz 2023-10-18T19:23:27.861 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93348.log 2023-10-18T19:23:27.861 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128310.log.gz 2023-10-18T19:23:27.861 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117635.log 2023-10-18T19:23:27.861 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95366.log.gz 2023-10-18T19:23:27.862 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117936.log 2023-10-18T19:23:27.862 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93348.log.gz 2023-10-18T19:23:27.862 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.117635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101670.log 2023-10-18T19:23:27.863 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117635.log.gz 2023-10-18T19:23:27.863 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.117936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99655.log 2023-10-18T19:23:27.863 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117936.log.gz 2023-10-18T19:23:27.863 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113904.log 2023-10-18T19:23:27.864 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101670.log.gz 2023-10-18T19:23:27.864 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108726.log 2023-10-18T19:23:27.864 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99655.log.gz 2023-10-18T19:23:27.865 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123327.log 2023-10-18T19:23:27.865 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113904.log.gz 2023-10-18T19:23:27.865 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119154.log 2023-10-18T19:23:27.865 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108726.log.gz 2023-10-18T19:23:27.866 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123327.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.123327.log.gz-5 2023-10-18T19:23:27.866 INFO:teuthology.orchestra.run.smithi082.stderr:--verbose -- /var/log/ceph/ceph-client.admin.119740.log 2023-10-18T19:23:27.866 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93692.log 2023-10-18T19:23:27.866 INFO:teuthology.orchestra.run.smithi082.stderr: 27.1% -- replaced with /var/log/ceph/ceph-client.admin.119154.log.gz 2023-10-18T19:23:27.867 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124364.log 2023-10-18T19:23:27.867 INFO:teuthology.orchestra.run.smithi082.stderr: 22.5% -- replaced with /var/log/ceph/ceph-client.admin.119740.log.gz 2023-10-18T19:23:27.867 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93692.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93692.log.gz 2023-10-18T19:23:27.868 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125443.log 2023-10-18T19:23:27.868 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110570.log 2023-10-18T19:23:27.868 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124364.log.gz 2023-10-18T19:23:27.868 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133100.log 2023-10-18T19:23:27.869 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125443.log.gz 2023-10-18T19:23:27.869 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110570.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110570.log.gz 2023-10-18T19:23:27.869 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127039.log 2023-10-18T19:23:27.870 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116967.log 2023-10-18T19:23:27.870 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133100.log.gz 2023-10-18T19:23:27.870 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131455.log 2023-10-18T19:23:27.871 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127039.log.gz 2023-10-18T19:23:27.871 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.116967.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116967.log.gz 2023-10-18T19:23:27.871 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102813.log 2023-10-18T19:23:27.871 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102708.log 2023-10-18T19:23:27.872 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131455.log.gz 2023-10-18T19:23:27.872 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135563.log 2023-10-18T19:23:27.872 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102813.log.gz 2023-10-18T19:23:27.873 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102708.log: 0.0%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.102708.log.gz--verbose 2023-10-18T19:23:27.873 INFO:teuthology.orchestra.run.smithi082.stderr:-- /var/log/ceph/ceph-client.admin.131126.log 2023-10-18T19:23:27.873 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140363.log 2023-10-18T19:23:27.873 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135563.log.gz 2023-10-18T19:23:27.874 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97790.log 2023-10-18T19:23:27.874 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131126.log.gz 2023-10-18T19:23:27.874 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.140363.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.140363.log.gzgzip 2023-10-18T19:23:27.875 INFO:teuthology.orchestra.run.smithi082.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.126535.log 2023-10-18T19:23:27.875 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132270.log 2023-10-18T19:23:27.875 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97790.log.gz 2023-10-18T19:23:27.875 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110656.log 2023-10-18T19:23:27.876 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126535.log.gz 2023-10-18T19:23:27.876 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132270.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132270.log.gz 2023-10-18T19:23:27.876 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106203.log 2023-10-18T19:23:27.876 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92380.log 2023-10-18T19:23:27.877 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110656.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110656.log.gz 2023-10-18T19:23:27.877 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139583.log 2023-10-18T19:23:27.877 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106203.log.gz 2023-10-18T19:23:27.878 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130938.log 2023-10-18T19:23:27.878 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.92380.log.gz 2023-10-18T19:23:27.878 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.139583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108470.log 2023-10-18T19:23:27.878 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.139583.log.gz 2023-10-18T19:23:27.879 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105032.log 2023-10-18T19:23:27.879 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130938.log.gz 2023-10-18T19:23:27.879 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113256.log 2023-10-18T19:23:27.879 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108470.log.gz 2023-10-18T19:23:27.880 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124798.log 2023-10-18T19:23:27.880 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105032.log.gz 2023-10-18T19:23:27.880 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131413.log 2023-10-18T19:23:27.881 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113256.log.gz 2023-10-18T19:23:27.881 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106523.log 2023-10-18T19:23:27.881 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124798.log.gz 2023-10-18T19:23:27.882 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123708.log 2023-10-18T19:23:27.882 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131413.log.gz 2023-10-18T19:23:27.883 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123905.log 2023-10-18T19:23:27.883 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106523.log.gz 2023-10-18T19:23:27.884 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134669.log 2023-10-18T19:23:27.884 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123708.log.gz 2023-10-18T19:23:27.885 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128133.log 2023-10-18T19:23:27.885 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123905.log.gz 2023-10-18T19:23:27.885 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99975.log 2023-10-18T19:23:27.886 INFO:teuthology.orchestra.run.smithi082.stderr: 55.9% -- replaced with /var/log/ceph/ceph-client.admin.134669.log.gz 2023-10-18T19:23:27.886 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130881.log 2023-10-18T19:23:27.887 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128133.log.gz 2023-10-18T19:23:27.887 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97960.log 2023-10-18T19:23:27.887 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99975.log.gz 2023-10-18T19:23:27.888 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99795.log 2023-10-18T19:23:27.888 INFO:teuthology.orchestra.run.smithi082.stderr: 28.2% -- replaced with /var/log/ceph/ceph-client.admin.130881.log.gz 2023-10-18T19:23:27.888 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108767.log 2023-10-18T19:23:27.888 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97960.log.gz 2023-10-18T19:23:27.889 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128534.log 2023-10-18T19:23:27.889 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99795.log.gz 2023-10-18T19:23:27.889 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92162.log 2023-10-18T19:23:27.890 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108767.log.gz 2023-10-18T19:23:27.890 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107544.log 2023-10-18T19:23:27.890 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128534.log.gz 2023-10-18T19:23:27.890 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137617.log 2023-10-18T19:23:27.891 INFO:teuthology.orchestra.run.smithi082.stderr: 92.6% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-18T19:23:27.891 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.92162.log.gz 2023-10-18T19:23:27.891 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110979.log 2023-10-18T19:23:27.892 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107544.log.gz 2023-10-18T19:23:27.892 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.137617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112473.log 2023-10-18T19:23:27.892 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.137617.log.gz 2023-10-18T19:23:27.892 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138712.log 2023-10-18T19:23:27.893 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110979.log.gz 2023-10-18T19:23:27.893 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112776.log 2023-10-18T19:23:27.893 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112473.log.gz 2023-10-18T19:23:27.894 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.138712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111437.log 2023-10-18T19:23:27.894 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.138712.log.gz 2023-10-18T19:23:27.894 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114421.log 2023-10-18T19:23:27.894 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112776.log.gz 2023-10-18T19:23:27.895 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109387.log 2023-10-18T19:23:27.895 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111437.log.gz 2023-10-18T19:23:27.895 INFO:teuthology.orchestra.run.smithi082.stderr:gzip /var/log/ceph/ceph-client.admin.114421.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.127997.log 2023-10-18T19:23:27.896 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114421.log.gz 2023-10-18T19:23:27.896 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103212.log 2023-10-18T19:23:27.896 INFO:teuthology.orchestra.run.smithi082.stderr: 23.6% -- replaced with /var/log/ceph/ceph-client.admin.109387.log.gz 2023-10-18T19:23:27.896 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138024.log 2023-10-18T19:23:27.897 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127997.log.gz 2023-10-18T19:23:27.897 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135582.log 2023-10-18T19:23:27.897 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103212.log.gz 2023-10-18T19:23:27.897 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.138024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138158.log 2023-10-18T19:23:27.898 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.138024.log.gz 2023-10-18T19:23:27.898 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127153.log 2023-10-18T19:23:27.898 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135582.log.gz 2023-10-18T19:23:27.898 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.138158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103653.log 2023-10-18T19:23:27.899 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.138158.log.gz 2023-10-18T19:23:27.899 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115525.log 2023-10-18T19:23:27.899 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127153.log.gz 2023-10-18T19:23:27.900 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139407.log 2023-10-18T19:23:27.900 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103653.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103653.log.gz 2023-10-18T19:23:27.900 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92000.log 2023-10-18T19:23:27.900 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115525.log.gz 2023-10-18T19:23:27.901 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.139407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131288.log 2023-10-18T19:23:27.901 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.139407.log.gz 2023-10-18T19:23:27.901 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92000.log: gzip 0.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.92000.log.gz--verbose 2023-10-18T19:23:27.901 INFO:teuthology.orchestra.run.smithi082.stderr:-- /var/log/ceph/ceph-client.admin.101398.log 2023-10-18T19:23:27.902 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141270.log 2023-10-18T19:23:27.902 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131288.log.gz 2023-10-18T19:23:27.902 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132033.log 2023-10-18T19:23:27.903 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101398.log.gz 2023-10-18T19:23:27.903 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.141270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131247.log 2023-10-18T19:23:27.903 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.141270.log.gz 2023-10-18T19:23:27.903 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118719.log 2023-10-18T19:23:27.904 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132033.log.gz 2023-10-18T19:23:27.904 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112320.log 2023-10-18T19:23:27.904 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131247.log.gz 2023-10-18T19:23:27.904 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131861.log 2023-10-18T19:23:27.905 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118719.log.gz 2023-10-18T19:23:27.905 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116149.log 2023-10-18T19:23:27.905 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112320.log.gz 2023-10-18T19:23:27.906 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119762.log 2023-10-18T19:23:27.906 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131861.log.gz 2023-10-18T19:23:27.906 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.116149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104430.log 2023-10-18T19:23:27.906 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116149.log.gz 2023-10-18T19:23:27.907 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95387.log 2023-10-18T19:23:27.907 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119762.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119762.log.gz 2023-10-18T19:23:27.907 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108946.log 2023-10-18T19:23:27.907 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104430.log.gz 2023-10-18T19:23:27.908 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140455.log 2023-10-18T19:23:27.908 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95387.log.gz 2023-10-18T19:23:27.908 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134027.log 2023-10-18T19:23:27.909 INFO:teuthology.orchestra.run.smithi082.stderr: 22.4% -- replaced with /var/log/ceph/ceph-client.admin.108946.log.gz 2023-10-18T19:23:27.909 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.140455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112084.log 2023-10-18T19:23:27.909 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.140455.log.gz 2023-10-18T19:23:27.909 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105118.log 2023-10-18T19:23:27.910 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134027.log.gz 2023-10-18T19:23:27.910 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99215.log 2023-10-18T19:23:27.910 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112084.log.gz 2023-10-18T19:23:27.911 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95430.log 2023-10-18T19:23:27.911 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105118.log.gz 2023-10-18T19:23:27.911 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134897.log 2023-10-18T19:23:27.911 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99215.log.gz 2023-10-18T19:23:27.912 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100752.log 2023-10-18T19:23:27.912 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95430.log.gz 2023-10-18T19:23:27.912 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133452.log 2023-10-18T19:23:27.913 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134897.log.gz 2023-10-18T19:23:27.913 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99695.log 2023-10-18T19:23:27.913 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100752.log.gz 2023-10-18T19:23:27.913 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.133452.log: -- /var/log/ceph/ceph-client.admin.100415.log 2023-10-18T19:23:27.914 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133452.log.gz 2023-10-18T19:23:27.914 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126937.log 2023-10-18T19:23:27.914 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99695.log.gz 2023-10-18T19:23:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134543.log 2023-10-18T19:23:27.915 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100415.log.gz 2023-10-18T19:23:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130473.log 2023-10-18T19:23:27.915 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126937.log.gz 2023-10-18T19:23:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132700.log 2023-10-18T19:23:27.916 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134543.log.gz 2023-10-18T19:23:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126748.log 2023-10-18T19:23:27.916 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130473.log.gz 2023-10-18T19:23:27.917 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102548.log 2023-10-18T19:23:27.917 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132700.log.gz 2023-10-18T19:23:27.917 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114894.log 2023-10-18T19:23:27.917 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126748.log.gz 2023-10-18T19:23:27.918 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102129.log 2023-10-18T19:23:27.918 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102548.log.gz 2023-10-18T19:23:27.918 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103464.log 2023-10-18T19:23:27.919 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114894.log.gz 2023-10-18T19:23:27.919 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93370.log 2023-10-18T19:23:27.919 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102129.log.gz 2023-10-18T19:23:27.919 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116127.log 2023-10-18T19:23:27.920 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103464.log.gz 2023-10-18T19:23:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95581.log 2023-10-18T19:23:27.920 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93370.log.gz 2023-10-18T19:23:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.116127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127976.log 2023-10-18T19:23:27.921 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116127.log.gz 2023-10-18T19:23:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114269.log 2023-10-18T19:23:27.921 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95581.log.gz 2023-10-18T19:23:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130067.log 2023-10-18T19:23:27.922 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127976.log.gz 2023-10-18T19:23:27.922 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92552.log 2023-10-18T19:23:27.922 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114269.log.gz 2023-10-18T19:23:27.923 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102981.log 2023-10-18T19:23:27.923 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130067.log.gz 2023-10-18T19:23:27.923 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125422.log 2023-10-18T19:23:27.923 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92552.log.gz 2023-10-18T19:23:27.924 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141102.log 2023-10-18T19:23:27.924 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102981.log.gz 2023-10-18T19:23:27.924 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105474.log 2023-10-18T19:23:27.924 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125422.log.gz 2023-10-18T19:23:27.925 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.141102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107794.log 2023-10-18T19:23:27.925 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.141102.log.gz 2023-10-18T19:23:27.925 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96228.log 2023-10-18T19:23:27.926 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105474.log.gz 2023-10-18T19:23:27.926 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121025.log 2023-10-18T19:23:27.926 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107794.log.gz 2023-10-18T19:23:27.926 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93971.log 2023-10-18T19:23:27.927 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96228.log.gz 2023-10-18T19:23:27.927 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108169.log 2023-10-18T19:23:27.927 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121025.log.gz 2023-10-18T19:23:27.927 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108709.log 2023-10-18T19:23:27.928 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93971.log.gz 2023-10-18T19:23:27.928 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96400.log 2023-10-18T19:23:27.928 INFO:teuthology.orchestra.run.smithi082.stderr: 23.3% -- replaced with /var/log/ceph/ceph-client.admin.108169.log.gz 2023-10-18T19:23:27.929 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115296.log 2023-10-18T19:23:27.929 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.108709.log.gz 2023-10-18T19:23:27.929 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138628.log 2023-10-18T19:23:27.929 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96400.log.gz 2023-10-18T19:23:27.930 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111783.log 2023-10-18T19:23:27.930 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115296.log.gz 2023-10-18T19:23:27.930 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.138628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118476.log 2023-10-18T19:23:27.931 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.138628.log.gz 2023-10-18T19:23:27.931 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91377.log 2023-10-18T19:23:27.931 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111783.log.gz 2023-10-18T19:23:27.931 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106485.log 2023-10-18T19:23:27.932 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118476.log.gz 2023-10-18T19:23:27.932 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123026.log 2023-10-18T19:23:27.932 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91377.log.gz 2023-10-18T19:23:27.932 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98402.log 2023-10-18T19:23:27.933 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.106485.log.gz 2023-10-18T19:23:27.933 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108062.log 2023-10-18T19:23:27.933 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123026.log.gz 2023-10-18T19:23:27.934 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99595.log 2023-10-18T19:23:27.934 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.98402.log.gz 2023-10-18T19:23:27.934 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113564.log 2023-10-18T19:23:27.935 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108062.log.gz 2023-10-18T19:23:27.935 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107232.log 2023-10-18T19:23:27.935 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99595.log.gz 2023-10-18T19:23:27.935 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126323.log 2023-10-18T19:23:27.936 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113564.log.gz 2023-10-18T19:23:27.936 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107774.log 2023-10-18T19:23:27.936 INFO:teuthology.orchestra.run.smithi082.stderr: 54.0% -- replaced with /var/log/ceph/ceph-client.admin.107232.log.gz 2023-10-18T19:23:27.936 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119276.log 2023-10-18T19:23:27.937 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126323.log.gz 2023-10-18T19:23:27.937 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100922.log 2023-10-18T19:23:27.937 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107774.log.gz 2023-10-18T19:23:27.937 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91485.log 2023-10-18T19:23:27.938 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119276.log.gz 2023-10-18T19:23:27.938 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105658.log 2023-10-18T19:23:27.938 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100922.log.gz 2023-10-18T19:23:27.939 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135026.log 2023-10-18T19:23:27.939 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91485.log.gz 2023-10-18T19:23:27.939 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120666.log 2023-10-18T19:23:27.939 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105658.log.gz 2023-10-18T19:23:27.940 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131797.log 2023-10-18T19:23:27.940 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135026.log.gz 2023-10-18T19:23:27.940 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128373.log 2023-10-18T19:23:27.941 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120666.log.gz 2023-10-18T19:23:27.941 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92201.log 2023-10-18T19:23:27.941 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131797.log.gz 2023-10-18T19:23:27.941 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128373.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128373.log.gz 2023-10-18T19:23:27.942 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112553.log 2023-10-18T19:23:27.942 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92201.log: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.92201.log.gz 2023-10-18T19:23:27.942 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131775.log 2023-10-18T19:23:27.943 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106439.log 2023-10-18T19:23:27.943 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112553.log.gz 2023-10-18T19:23:27.943 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131775.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-18T19:23:27.943 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131775.log.gz 2023-10-18T19:23:27.944 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129929.log 2023-10-18T19:23:27.944 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106439.log.gz 2023-10-18T19:23:27.944 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118739.log 2023-10-18T19:23:27.944 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112656.log 2023-10-18T19:23:27.945 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129929.log.gz 2023-10-18T19:23:27.945 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97025.log 2023-10-18T19:23:27.945 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118739.log.gz 2023-10-18T19:23:27.946 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112656.log: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-18T19:23:27.946 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102401.log 2023-10-18T19:23:27.946 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.112656.log.gz 2023-10-18T19:23:27.947 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142210.log 2023-10-18T19:23:27.947 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97025.log.gz 2023-10-18T19:23:27.947 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135263.log 2023-10-18T19:23:27.947 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102401.log.gz 2023-10-18T19:23:27.948 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.142210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134307.log 2023-10-18T19:23:27.948 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.142210.log.gz 2023-10-18T19:23:27.948 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120149.log 2023-10-18T19:23:27.949 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135263.log.gz 2023-10-18T19:23:27.949 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92570.log 2023-10-18T19:23:27.949 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134307.log.gz 2023-10-18T19:23:27.949 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112533.log 2023-10-18T19:23:27.950 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120149.log.gz 2023-10-18T19:23:27.950 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114378.log 2023-10-18T19:23:27.950 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.92570.log.gz 2023-10-18T19:23:27.951 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138670.log 2023-10-18T19:23:27.951 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112533.log.gz 2023-10-18T19:23:27.951 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115483.log 2023-10-18T19:23:27.951 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114378.log.gz 2023-10-18T19:23:27.952 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.138670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105075.log 2023-10-18T19:23:27.952 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.138670.log.gz 2023-10-18T19:23:27.952 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103023.log 2023-10-18T19:23:27.952 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115483.log.gz 2023-10-18T19:23:27.953 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100295.log 2023-10-18T19:23:27.953 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105075.log.gz 2023-10-18T19:23:27.953 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114355.log 2023-10-18T19:23:27.954 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103023.log.gz 2023-10-18T19:23:27.954 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124544.log 2023-10-18T19:23:27.954 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100295.log.gz 2023-10-18T19:23:27.954 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130667.log 2023-10-18T19:23:27.955 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114355.log.gz 2023-10-18T19:23:27.955 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117467.log 2023-10-18T19:23:27.955 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124544.log.gz 2023-10-18T19:23:27.955 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93671.log 2023-10-18T19:23:27.956 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130667.log.gz 2023-10-18T19:23:27.956 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.117467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100275.log 2023-10-18T19:23:27.956 INFO:teuthology.orchestra.run.smithi082.stderr: 54.3% -- replaced with /var/log/ceph/ceph-client.admin.117467.log.gz 2023-10-18T19:23:27.956 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126707.log 2023-10-18T19:23:27.957 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93671.log.gz 2023-10-18T19:23:27.957 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107030.log 2023-10-18T19:23:27.957 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100275.log.gz 2023-10-18T19:23:27.958 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124777.log 2023-10-18T19:23:27.958 INFO:teuthology.orchestra.run.smithi082.stderr: 50.7% -- replaced with /var/log/ceph/ceph-client.admin.126707.log.gz 2023-10-18T19:23:27.958 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99675.log 2023-10-18T19:23:27.958 INFO:teuthology.orchestra.run.smithi082.stderr: 22.4% -- replaced with /var/log/ceph/ceph-client.admin.107030.log.gz 2023-10-18T19:23:27.959 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114485.log 2023-10-18T19:23:27.959 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124777.log.gz 2023-10-18T19:23:27.959 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122940.log 2023-10-18T19:23:27.960 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99675.log.gz 2023-10-18T19:23:27.960 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127464.log 2023-10-18T19:23:27.960 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114485.log.gz 2023-10-18T19:23:27.960 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111499.log 2023-10-18T19:23:27.961 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122940.log.gz 2023-10-18T19:23:27.961 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122961.log 2023-10-18T19:23:27.961 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127464.log.gz 2023-10-18T19:23:27.961 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134221.log 2023-10-18T19:23:27.962 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111499.log.gz 2023-10-18T19:23:27.962 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104795.log 2023-10-18T19:23:27.962 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122961.log.gz 2023-10-18T19:23:27.963 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97382.log 2023-10-18T19:23:27.963 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134221.log.gz 2023-10-18T19:23:27.963 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106074.log 2023-10-18T19:23:27.963 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104795.log.gz 2023-10-18T19:23:27.964 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129494.log 2023-10-18T19:23:27.964 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97382.log.gz 2023-10-18T19:23:27.964 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106544.log 2023-10-18T19:23:27.964 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106074.log.gz 2023-10-18T19:23:27.965 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109836.log 2023-10-18T19:23:27.965 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129494.log.gz 2023-10-18T19:23:27.965 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112148.log 2023-10-18T19:23:27.966 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106544.log.gz 2023-10-18T19:23:27.966 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114808.log 2023-10-18T19:23:27.966 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109836.log.gz 2023-10-18T19:23:27.966 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108002.log 2023-10-18T19:23:27.967 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112148.log.gz 2023-10-18T19:23:27.967 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105380.log 2023-10-18T19:23:27.967 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114808.log.gz 2023-10-18T19:23:27.967 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108002.log: gzip 0.0%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.108002.log.gz-- 2023-10-18T19:23:27.968 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128706.log 2023-10-18T19:23:27.968 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105380.log: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.105380.log.gz 2023-10-18T19:23:27.968 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125916.log 2023-10-18T19:23:27.968 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109513.log 2023-10-18T19:23:27.969 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128706.log.gz 2023-10-18T19:23:27.969 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124464.log 2023-10-18T19:23:27.969 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125916.log.gz 2023-10-18T19:23:27.969 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126171.log 2023-10-18T19:23:27.970 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109513.log.gz 2023-10-18T19:23:27.970 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132076.log 2023-10-18T19:23:27.970 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124464.log.gz 2023-10-18T19:23:27.971 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94011.log 2023-10-18T19:23:27.971 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126171.log.gz 2023-10-18T19:23:27.971 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91936.log 2023-10-18T19:23:27.973 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132076.log.gz 2023-10-18T19:23:27.973 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106627.log 2023-10-18T19:23:27.973 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94011.log.gz 2023-10-18T19:23:27.974 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123745.log 2023-10-18T19:23:27.974 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91936.log.gz 2023-10-18T19:23:27.974 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97416.log 2023-10-18T19:23:27.975 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106627.log.gz 2023-10-18T19:23:27.975 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135521.log 2023-10-18T19:23:27.975 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123745.log.gz 2023-10-18T19:23:27.975 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105267.log 2023-10-18T19:23:27.976 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.97416.log.gz 2023-10-18T19:23:27.976 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138755.log 2023-10-18T19:23:27.976 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135521.log.gz 2023-10-18T19:23:27.976 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121318.log 2023-10-18T19:23:27.977 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.105267.log.gz 2023-10-18T19:23:27.977 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.138755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130774.log 2023-10-18T19:23:27.977 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.138755.log.gz 2023-10-18T19:23:27.977 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109249.log 2023-10-18T19:23:27.978 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121318.log.gz 2023-10-18T19:23:27.978 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124177.log 2023-10-18T19:23:27.978 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130774.log.gz 2023-10-18T19:23:27.979 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98975.log 2023-10-18T19:23:27.979 INFO:teuthology.orchestra.run.smithi082.stderr: 22.1% -- replaced with /var/log/ceph/ceph-client.admin.109249.log.gz 2023-10-18T19:23:27.979 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122025.log 2023-10-18T19:23:27.979 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124177.log.gz 2023-10-18T19:23:27.980 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103737.log 2023-10-18T19:23:27.980 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98975.log.gz 2023-10-18T19:23:27.980 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106668.log 2023-10-18T19:23:27.981 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122025.log.gz 2023-10-18T19:23:27.981 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112105.log 2023-10-18T19:23:27.981 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103737.log.gz 2023-10-18T19:23:27.981 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108845.log 2023-10-18T19:23:27.982 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106668.log.gz 2023-10-18T19:23:27.982 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96185.log 2023-10-18T19:23:27.982 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112105.log.gz 2023-10-18T19:23:27.982 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110355.log 2023-10-18T19:23:27.983 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108845.log.gz 2023-10-18T19:23:27.983 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132915.log 2023-10-18T19:23:27.983 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96185.log.gz 2023-10-18T19:23:27.983 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96019.log 2023-10-18T19:23:27.984 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110355.log.gz 2023-10-18T19:23:27.984 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94828.log 2023-10-18T19:23:27.984 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132915.log.gz 2023-10-18T19:23:27.985 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133553.log 2023-10-18T19:23:27.985 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96019.log.gz 2023-10-18T19:23:27.985 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140019.log 2023-10-18T19:23:27.986 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94828.log.gz 2023-10-18T19:23:27.986 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107072.log 2023-10-18T19:23:27.986 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133553.log.gz 2023-10-18T19:23:27.986 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.140019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118801.log 2023-10-18T19:23:27.987 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.140019.log.gz 2023-10-18T19:23:27.987 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126378.log 2023-10-18T19:23:27.987 INFO:teuthology.orchestra.run.smithi082.stderr: 23.2% -- replaced with /var/log/ceph/ceph-client.admin.107072.log.gz 2023-10-18T19:23:27.988 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90106.log 2023-10-18T19:23:27.988 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118801.log.gz 2023-10-18T19:23:27.988 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125658.log 2023-10-18T19:23:27.988 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126378.log.gz 2023-10-18T19:23:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.90106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121552.log 2023-10-18T19:23:27.989 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90106.log.gz 2023-10-18T19:23:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99135.log 2023-10-18T19:23:27.989 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125658.log.gz 2023-10-18T19:23:27.990 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104052.log 2023-10-18T19:23:27.990 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121552.log.gz 2023-10-18T19:23:27.990 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97348.log 2023-10-18T19:23:27.990 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99135.log.gz 2023-10-18T19:23:27.991 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129834.log 2023-10-18T19:23:27.991 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104052.log.gz 2023-10-18T19:23:27.991 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95904.log 2023-10-18T19:23:27.991 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97348.log.gz 2023-10-18T19:23:27.992 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104602.log 2023-10-18T19:23:27.992 INFO:teuthology.orchestra.run.smithi082.stderr: 56.9% -- replaced with /var/log/ceph/ceph-client.admin.129834.log.gz 2023-10-18T19:23:27.992 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90738.log 2023-10-18T19:23:27.993 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95904.log.gz 2023-10-18T19:23:27.993 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128628.log 2023-10-18T19:23:27.993 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104602.log.gz 2023-10-18T19:23:27.993 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.90738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95794.log 2023-10-18T19:23:27.994 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90738.log.gz 2023-10-18T19:23:27.994 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96701.log 2023-10-18T19:23:27.995 INFO:teuthology.orchestra.run.smithi082.stderr: 50.7% -- replaced with /var/log/ceph/ceph-client.admin.128628.log.gz 2023-10-18T19:23:27.995 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137940.log 2023-10-18T19:23:27.995 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95794.log.gz 2023-10-18T19:23:27.995 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92144.log 2023-10-18T19:23:27.996 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96701.log.gz 2023-10-18T19:23:27.996 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.137940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118538.log 2023-10-18T19:23:27.996 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.137940.log.gz 2023-10-18T19:23:27.996 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107484.log 2023-10-18T19:23:27.997 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92144.log.gz 2023-10-18T19:23:27.997 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119805.log 2023-10-18T19:23:27.997 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118538.log.gz 2023-10-18T19:23:27.998 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139893.log 2023-10-18T19:23:27.998 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107484.log.gz 2023-10-18T19:23:27.998 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111402.log 2023-10-18T19:23:27.998 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119805.log.gz 2023-10-18T19:23:27.999 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.139893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129185.log 2023-10-18T19:23:27.999 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.139893.log.gz 2023-10-18T19:23:27.999 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111298.log 2023-10-18T19:23:28.000 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111402.log.gz 2023-10-18T19:23:28.000 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103590.log 2023-10-18T19:23:28.000 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129185.log.gz 2023-10-18T19:23:28.000 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122418.log 2023-10-18T19:23:28.001 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111298.log.gz 2023-10-18T19:23:28.001 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111066.log 2023-10-18T19:23:28.001 INFO:teuthology.orchestra.run.smithi082.stderr: 22.5% -- replaced with /var/log/ceph/ceph-client.admin.103590.log.gz 2023-10-18T19:23:28.001 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121172.log 2023-10-18T19:23:28.002 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122418.log.gz 2023-10-18T19:23:28.002 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138376.log 2023-10-18T19:23:28.002 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111066.log.gz 2023-10-18T19:23:28.003 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117487.log 2023-10-18T19:23:28.003 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121172.log.gz 2023-10-18T19:23:28.003 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.138376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131517.log 2023-10-18T19:23:28.003 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.138376.log.gz 2023-10-18T19:23:28.004 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.117487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103233.log 2023-10-18T19:23:28.004 INFO:teuthology.orchestra.run.smithi082.stderr: 56.1% -- replaced with /var/log/ceph/ceph-client.admin.117487.log.gz 2023-10-18T19:23:28.004 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132141.log 2023-10-18T19:23:28.005 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131517.log.gz 2023-10-18T19:23:28.005 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96593.log 2023-10-18T19:23:28.005 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103233.log.gz 2023-10-18T19:23:28.005 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101228.log 2023-10-18T19:23:28.006 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132141.log.gz 2023-10-18T19:23:28.006 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142820.log 2023-10-18T19:23:28.006 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96593.log.gz 2023-10-18T19:23:28.006 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115891.log 2023-10-18T19:23:28.007 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101228.log.gz 2023-10-18T19:23:28.007 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.142820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106710.log 2023-10-18T19:23:28.007 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.142820.log.gz 2023-10-18T19:23:28.007 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124841.log 2023-10-18T19:23:28.008 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115891.log.gz 2023-10-18T19:23:28.008 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111263.log 2023-10-18T19:23:28.008 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106710.log.gz 2023-10-18T19:23:28.009 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101092.log 2023-10-18T19:23:28.009 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124841.log.gz 2023-10-18T19:23:28.009 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124421.log 2023-10-18T19:23:28.009 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111263.log.gz 2023-10-18T19:23:28.010 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97994.log 2023-10-18T19:23:28.010 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101092.log.gz 2023-10-18T19:23:28.010 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115977.log 2023-10-18T19:23:28.011 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124421.log.gz 2023-10-18T19:23:28.011 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134811.log 2023-10-18T19:23:28.011 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97994.log.gz 2023-10-18T19:23:28.011 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123668.log 2023-10-18T19:23:28.012 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115977.log.gz 2023-10-18T19:23:28.012 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95946.log 2023-10-18T19:23:28.012 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134811.log.gz 2023-10-18T19:23:28.013 INFO:teuthology.orchestra.run.smithi082.stderr:gzip /var/log/ceph/ceph-client.admin.123668.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.102527.log 2023-10-18T19:23:28.013 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123668.log.gz 2023-10-18T19:23:28.013 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119029.log 2023-10-18T19:23:28.013 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95946.log.gz 2023-10-18T19:23:28.014 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120795.log 2023-10-18T19:23:28.014 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102527.log.gz 2023-10-18T19:23:28.014 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99715.log 2023-10-18T19:23:28.015 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119029.log.gz 2023-10-18T19:23:28.015 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92835.log 2023-10-18T19:23:28.015 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120795.log.gz 2023-10-18T19:23:28.015 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132657.log 2023-10-18T19:23:28.016 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99715.log.gz 2023-10-18T19:23:28.016 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92942.log 2023-10-18T19:23:28.016 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92835.log.gz 2023-10-18T19:23:28.016 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111581.log 2023-10-18T19:23:28.017 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132657.log.gz 2023-10-18T19:23:28.017 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107630.log 2023-10-18T19:23:28.017 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92942.log.gz 2023-10-18T19:23:28.018 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132635.log 2023-10-18T19:23:28.018 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111581.log.gz 2023-10-18T19:23:28.018 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104283.log 2023-10-18T19:23:28.018 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107630.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107630.log.gz 2023-10-18T19:23:28.019 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95733.log 2023-10-18T19:23:28.019 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132635.log.gz 2023-10-18T19:23:28.019 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124118.log 2023-10-18T19:23:28.019 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104283.log.gz 2023-10-18T19:23:28.020 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97773.log 2023-10-18T19:23:28.020 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95733.log.gz 2023-10-18T19:23:28.020 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115998.log 2023-10-18T19:23:28.021 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124118.log.gz 2023-10-18T19:23:28.021 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 /var/log/ceph/ceph-client.admin.97773.log: --verbose -- /var/log/ceph/ceph-client.admin.131711.log 2023-10-18T19:23:28.021 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97773.log.gz 2023-10-18T19:23:28.021 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95323.log 2023-10-18T19:23:28.022 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115998.log.gz 2023-10-18T19:23:28.022 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124648.log 2023-10-18T19:23:28.022 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131711.log.gz 2023-10-18T19:23:28.023 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107113.log 2023-10-18T19:23:28.023 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95323.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95323.log.gz 2023-10-18T19:23:28.023 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126958.log 2023-10-18T19:23:28.023 INFO:teuthology.orchestra.run.smithi082.stderr: 24.3% -- replaced with /var/log/ceph/ceph-client.admin.124648.log.gz 2023-10-18T19:23:28.024 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119217.log 2023-10-18T19:23:28.024 INFO:teuthology.orchestra.run.smithi082.stderr: 23.6% -- replaced with /var/log/ceph/ceph-client.admin.107113.log.gz 2023-10-18T19:23:28.024 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124060.log /var/log/ceph/ceph-client.admin.126958.log: 2023-10-18T19:23:28.024 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126958.log.gz 2023-10-18T19:23:28.025 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125529.log 2023-10-18T19:23:28.025 INFO:teuthology.orchestra.run.smithi082.stderr: 22.5% -- replaced with /var/log/ceph/ceph-client.admin.119217.log.gz 2023-10-18T19:23:28.025 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105096.log 2023-10-18T19:23:28.025 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124060.log.gz 2023-10-18T19:23:28.026 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92860.log 2023-10-18T19:23:28.026 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125529.log.gz 2023-10-18T19:23:28.026 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125873.log 2023-10-18T19:23:28.027 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105096.log.gz 2023-10-18T19:23:28.027 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128493.log 2023-10-18T19:23:28.027 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92860.log.gz 2023-10-18T19:23:28.027 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113627.log 2023-10-18T19:23:28.028 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125873.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125873.log.gz 2023-10-18T19:23:28.028 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101415.log 2023-10-18T19:23:28.028 INFO:teuthology.orchestra.run.smithi082.stderr: 54.2% -- replaced with /var/log/ceph/ceph-client.admin.128493.log.gz 2023-10-18T19:23:28.028 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91829.log 2023-10-18T19:23:28.029 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113627.log.gz 2023-10-18T19:23:28.029 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.101415.log: -- /var/log/ceph/ceph-client.admin.111315.log 2023-10-18T19:23:28.029 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101415.log.gz 2023-10-18T19:23:28.029 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106311.log 2023-10-18T19:23:28.030 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91829.log.gz 2023-10-18T19:23:28.030 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100575.log 2023-10-18T19:23:28.030 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111315.log.gz 2023-10-18T19:23:28.031 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119069.log 2023-10-18T19:23:28.031 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106311.log.gz 2023-10-18T19:23:28.031 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96807.log 2023-10-18T19:23:28.031 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100575.log.gz 2023-10-18T19:23:28.032 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119069.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-18T19:23:28.032 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119069.log.gz 2023-10-18T19:23:28.032 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135091.log 2023-10-18T19:23:28.032 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96807.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96807.log.gz 2023-10-18T19:23:28.033 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104924.log 2023-10-18T19:23:28.033 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102231.log 2023-10-18T19:23:28.033 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135091.log.gz 2023-10-18T19:23:28.034 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96957.log 2023-10-18T19:23:28.034 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104924.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104924.log.gz 2023-10-18T19:23:28.034 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128056.log 2023-10-18T19:23:28.035 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102231.log.gz 2023-10-18T19:23:28.035 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98795.log 2023-10-18T19:23:28.035 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.96957.log.gz 2023-10-18T19:23:28.035 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97178.log 2023-10-18T19:23:28.036 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128056.log.gz 2023-10-18T19:23:28.036 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126067.log 2023-10-18T19:23:28.036 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98795.log.gz 2023-10-18T19:23:28.036 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133226.log 2023-10-18T19:23:28.037 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.97178.log.gz 2023-10-18T19:23:28.037 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87260.log 2023-10-18T19:23:28.037 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126067.log.gz 2023-10-18T19:23:28.038 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118616.log 2023-10-18T19:23:28.038 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133226.log.gz 2023-10-18T19:23:28.039 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.87260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112406.log 2023-10-18T19:23:28.039 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87260.log.gz 2023-10-18T19:23:28.039 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113070.log 2023-10-18T19:23:28.040 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118616.log.gz 2023-10-18T19:23:28.040 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101704.log 2023-10-18T19:23:28.041 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112406.log.gz 2023-10-18T19:23:28.041 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123413.log 2023-10-18T19:23:28.042 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113070.log.gz 2023-10-18T19:23:28.042 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131689.log 2023-10-18T19:23:28.043 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101704.log.gz 2023-10-18T19:23:28.043 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112946.log 2023-10-18T19:23:28.044 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123413.log.gz 2023-10-18T19:23:28.044 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124626.log 2023-10-18T19:23:28.045 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131689.log.gz 2023-10-18T19:23:28.045 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108589.log 2023-10-18T19:23:28.046 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112946.log.gz 2023-10-18T19:23:28.046 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115109.log 2023-10-18T19:23:28.047 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124626.log.gz 2023-10-18T19:23:28.047 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105204.log 2023-10-18T19:23:28.048 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.108589.log.gz 2023-10-18T19:23:28.048 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115232.log 2023-10-18T19:23:28.048 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115109.log.gz 2023-10-18T19:23:28.048 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103611.log 2023-10-18T19:23:28.049 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105204.log.gz 2023-10-18T19:23:28.049 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115232.log: gzip 0.0%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.115232.log.gz-- 2023-10-18T19:23:28.049 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120902.log 2023-10-18T19:23:28.050 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99915.log 2023-10-18T19:23:28.050 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103611.log.gz 2023-10-18T19:23:28.050 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106010.log 2023-10-18T19:23:28.051 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120902.log.gz 2023-10-18T19:23:28.051 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99915.log: gzip -5 --verbose 0.0%-- /var/log/ceph/ceph-client.admin.121659.log -- replaced with /var/log/ceph/ceph-client.admin.99915.log.gz 2023-10-18T19:23:28.051 INFO:teuthology.orchestra.run.smithi082.stderr: 2023-10-18T19:23:28.051 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87122.log 2023-10-18T19:23:28.052 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106010.log.gz 2023-10-18T19:23:28.052 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134264.log 2023-10-18T19:23:28.052 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121659.log.gz 2023-10-18T19:23:28.052 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.87122.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87122.log.gz 2023-10-18T19:23:28.053 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110484.log 2023-10-18T19:23:28.053 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103380.log 2023-10-18T19:23:28.053 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134264.log.gz 2023-10-18T19:23:28.054 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100735.log 2023-10-18T19:23:28.054 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110484.log.gz 2023-10-18T19:23:28.054 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121152.log 2023-10-18T19:23:28.054 INFO:teuthology.orchestra.run.smithi082.stderr: 22.9% -- replaced with /var/log/ceph/ceph-client.admin.103380.log.gz 2023-10-18T19:23:28.055 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95043.log 2023-10-18T19:23:28.055 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100735.log.gz 2023-10-18T19:23:28.055 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96001.log 2023-10-18T19:23:28.056 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121152.log.gz 2023-10-18T19:23:28.056 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95043.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95043.log.gz 2023-10-18T19:23:28.056 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96974.log 2023-10-18T19:23:28.056 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129715.log 2023-10-18T19:23:28.057 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96001.log.gz 2023-10-18T19:23:28.057 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110158.log 2023-10-18T19:23:28.057 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.96974.log.gz 2023-10-18T19:23:28.058 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93112.log 2023-10-18T19:23:28.058 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129715.log.gz 2023-10-18T19:23:28.058 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115462.log 2023-10-18T19:23:28.058 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110158.log.gz 2023-10-18T19:23:28.059 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101500.log 2023-10-18T19:23:28.059 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93112.log.gz 2023-10-18T19:23:28.059 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104367.log 2023-10-18T19:23:28.060 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115462.log.gz 2023-10-18T19:23:28.060 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109578.log 2023-10-18T19:23:28.060 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101500.log.gz 2023-10-18T19:23:28.060 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124565.log 2023-10-18T19:23:28.061 INFO:teuthology.orchestra.run.smithi082.stderr: 23.3% -- replaced with /var/log/ceph/ceph-client.admin.104367.log.gz 2023-10-18T19:23:28.061 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111520.log 2023-10-18T19:23:28.061 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109578.log.gz 2023-10-18T19:23:28.061 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123391.log 2023-10-18T19:23:28.062 INFO:teuthology.orchestra.run.smithi082.stderr: 10.7% -- replaced with /var/log/ceph/ceph-client.admin.124565.log.gz 2023-10-18T19:23:28.062 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139155.log 2023-10-18T19:23:28.062 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111520.log.gz 2023-10-18T19:23:28.063 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123391.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123391.log.gz 2023-10-18T19:23:28.063 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142568.log 2023-10-18T19:23:28.063 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.139155.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.139155.log.gz 2023-10-18T19:23:28.063 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134649.log 2023-10-18T19:23:28.064 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.142568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142736.log 2023-10-18T19:23:28.064 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.142568.log.gz 2023-10-18T19:23:28.064 INFO:teuthology.orchestra.run.smithi082.stderr:gzip /var/log/ceph/ceph-client.admin.134649.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.131268.log 2023-10-18T19:23:28.065 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134649.log.gz 2023-10-18T19:23:28.065 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.142736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96658.log 2023-10-18T19:23:28.065 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.142736.log.gz 2023-10-18T19:23:28.065 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131268.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131268.log.gz 2023-10-18T19:23:28.066 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97586.log 2023-10-18T19:23:28.066 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112734.log 2023-10-18T19:23:28.066 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96658.log.gz 2023-10-18T19:23:28.067 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108082.log 2023-10-18T19:23:28.067 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.97586.log.gz 2023-10-18T19:23:28.067 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106988.log 2023-10-18T19:23:28.067 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112734.log.gz 2023-10-18T19:23:28.068 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104241.log 2023-10-18T19:23:28.068 INFO:teuthology.orchestra.run.smithi082.stderr: 54.5% -- replaced with /var/log/ceph/ceph-client.admin.108082.log.gz 2023-10-18T19:23:28.068 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128515.log 2023-10-18T19:23:28.069 INFO:teuthology.orchestra.run.smithi082.stderr: 23.2% -- replaced with /var/log/ceph/ceph-client.admin.106988.log.gz 2023-10-18T19:23:28.069 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125121.log 2023-10-18T19:23:28.070 INFO:teuthology.orchestra.run.smithi082.stderr: 23.7% -- replaced with /var/log/ceph/ceph-client.admin.104241.log.gz 2023-10-18T19:23:28.070 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106502.log 2023-10-18T19:23:28.070 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128515.log.gz 2023-10-18T19:23:28.070 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107051.log 2023-10-18T19:23:28.071 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125121.log.gz 2023-10-18T19:23:28.071 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102078.log 2023-10-18T19:23:28.071 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106502.log.gz 2023-10-18T19:23:28.071 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129991.log 2023-10-18T19:23:28.072 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107051.log.gz 2023-10-18T19:23:28.072 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97161.log 2023-10-18T19:23:28.072 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102078.log.gz 2023-10-18T19:23:28.073 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99775.log 2023-10-18T19:23:28.073 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129991.log.gz 2023-10-18T19:23:28.073 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90148.log 2023-10-18T19:23:28.074 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.97161.log.gz 2023-10-18T19:23:28.074 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123241.log 2023-10-18T19:23:28.075 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99775.log.gz 2023-10-18T19:23:28.075 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.90148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99235.log 2023-10-18T19:23:28.076 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90148.log.gz 2023-10-18T19:23:28.076 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112616.log 2023-10-18T19:23:28.077 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123241.log.gz 2023-10-18T19:23:28.077 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95640.log 2023-10-18T19:23:28.078 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99235.log.gz 2023-10-18T19:23:28.078 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120386.log 2023-10-18T19:23:28.078 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112616.log.gz 2023-10-18T19:23:28.079 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126340.log 2023-10-18T19:23:28.079 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.95640.log.gz 2023-10-18T19:23:28.080 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131646.log 2023-10-18T19:23:28.080 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120386.log.gz 2023-10-18T19:23:28.080 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112573.log 2023-10-18T19:23:28.080 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126340.log.gz 2023-10-18T19:23:28.081 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131063.log 2023-10-18T19:23:28.081 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131646.log.gz 2023-10-18T19:23:28.081 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130645.log 2023-10-18T19:23:28.082 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112573.log.gz 2023-10-18T19:23:28.082 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134113.log 2023-10-18T19:23:28.082 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131063.log.gz 2023-10-18T19:23:28.082 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124081.log 2023-10-18T19:23:28.083 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130645.log.gz 2023-10-18T19:23:28.083 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119133.log 2023-10-18T19:23:28.083 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134113.log.gz 2023-10-18T19:23:28.084 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95237.log 2023-10-18T19:23:28.084 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124081.log.gz 2023-10-18T19:23:28.084 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98368.log 2023-10-18T19:23:28.084 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119133.log.gz 2023-10-18T19:23:28.085 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134414.log 2023-10-18T19:23:28.085 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95237.log.gz 2023-10-18T19:23:28.085 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128037.log 2023-10-18T19:23:28.085 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.98368.log.gz 2023-10-18T19:23:28.086 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124712.log 2023-10-18T19:23:28.086 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134414.log.gz 2023-10-18T19:23:28.086 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128037.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128037.log.gz 2023-10-18T19:23:28.087 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97858.log 2023-10-18T19:23:28.087 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124712.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124712.log.gz 2023-10-18T19:23:28.087 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130839.log 2023-10-18T19:23:28.087 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130624.log 2023-10-18T19:23:28.088 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97858.log.gz 2023-10-18T19:23:28.088 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130839.log: /var/log/ceph/ceph-client.admin.120537.log 2023-10-18T19:23:28.088 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130839.log.gz 2023-10-18T19:23:28.088 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111350.log 2023-10-18T19:23:28.089 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130624.log.gz 2023-10-18T19:23:28.089 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120537.log: gzip -5 --verbose -- 0.0%/var/log/ceph/ceph-client.admin.112385.log 2023-10-18T19:23:28.089 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.120537.log.gz 2023-10-18T19:23:28.090 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108987.log 2023-10-18T19:23:28.090 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111350.log.gz 2023-10-18T19:23:28.090 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133390.log 2023-10-18T19:23:28.090 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112385.log.gz 2023-10-18T19:23:28.091 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100820.log 2023-10-18T19:23:28.091 INFO:teuthology.orchestra.run.smithi082.stderr: 22.0% -- replaced with /var/log/ceph/ceph-client.admin.108987.log.gz 2023-10-18T19:23:28.091 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134092.log 2023-10-18T19:23:28.092 INFO:teuthology.orchestra.run.smithi082.stderr: 22.0% -- replaced with /var/log/ceph/ceph-client.admin.133390.log.gz 2023-10-18T19:23:28.092 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99415.log 2023-10-18T19:23:28.092 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100820.log.gz 2023-10-18T19:23:28.092 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132162.log 2023-10-18T19:23:28.093 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134092.log.gz 2023-10-18T19:23:28.093 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135005.log 2023-10-18T19:23:28.093 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99415.log.gz 2023-10-18T19:23:28.093 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95516.log 2023-10-18T19:23:28.094 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132162.log.gz 2023-10-18T19:23:28.094 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135005.log: gzip 0.0%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.135005.log.gz-- 2023-10-18T19:23:28.094 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126879.log 2023-10-18T19:23:28.095 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132098.log 2023-10-18T19:23:28.095 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95516.log.gz 2023-10-18T19:23:28.095 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87162.log 2023-10-18T19:23:28.096 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126879.log.gz 2023-10-18T19:23:28.096 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132098.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132098.log.gz 2023-10-18T19:23:28.096 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119826.log 2023-10-18T19:23:28.097 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.87162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108887.log 2023-10-18T19:23:28.097 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87162.log.gz 2023-10-18T19:23:28.097 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109087.log 2023-10-18T19:23:28.097 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119826.log.gz 2023-10-18T19:23:28.098 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135435.log 2023-10-18T19:23:28.098 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108887.log.gz 2023-10-18T19:23:28.098 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109370.log 2023-10-18T19:23:28.098 INFO:teuthology.orchestra.run.smithi082.stderr: 31.5% -- replaced with /var/log/ceph/ceph-client.admin.109087.log.gz 2023-10-18T19:23:28.099 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127921.log 2023-10-18T19:23:28.099 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135435.log.gz 2023-10-18T19:23:28.099 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109771.log 2023-10-18T19:23:28.100 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109370.log.gz 2023-10-18T19:23:28.100 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115210.log 2023-10-18T19:23:28.100 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127921.log.gz 2023-10-18T19:23:28.101 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103842.log 2023-10-18T19:23:28.101 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109771.log.gz 2023-10-18T19:23:28.101 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106053.log 2023-10-18T19:23:28.101 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115210.log.gz 2023-10-18T19:23:28.102 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117850.log 2023-10-18T19:23:28.102 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103842.log.gz 2023-10-18T19:23:28.102 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123434.log 2023-10-18T19:23:28.103 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106053.log.gz 2023-10-18T19:23:28.103 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.117850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103695.log 2023-10-18T19:23:28.103 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117850.log.gz 2023-10-18T19:23:28.103 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130323.log 2023-10-18T19:23:28.104 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123434.log.gz 2023-10-18T19:23:28.104 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119676.log 2023-10-18T19:23:28.104 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103695.log.gz 2023-10-18T19:23:28.105 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93434.log 2023-10-18T19:23:28.105 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130323.log.gz 2023-10-18T19:23:28.105 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97297.log 2023-10-18T19:23:28.105 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119676.log.gz 2023-10-18T19:23:28.106 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140321.log 2023-10-18T19:23:28.106 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93434.log.gz 2023-10-18T19:23:28.106 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98147.log 2023-10-18T19:23:28.106 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97297.log.gz 2023-10-18T19:23:28.107 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.140321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135198.log 2023-10-18T19:23:28.107 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.140321.log.gz 2023-10-18T19:23:28.107 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108230.log 2023-10-18T19:23:28.108 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.98147.log.gz 2023-10-18T19:23:28.108 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102506.log 2023-10-18T19:23:28.108 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135198.log.gz 2023-10-18T19:23:28.108 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130495.log 2023-10-18T19:23:28.109 INFO:teuthology.orchestra.run.smithi082.stderr: 22.8% -- replaced with /var/log/ceph/ceph-client.admin.108230.log.gz 2023-10-18T19:23:28.109 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130581.log 2023-10-18T19:23:28.109 INFO:teuthology.orchestra.run.smithi082.stderr: 22.8% -- replaced with /var/log/ceph/ceph-client.admin.102506.log.gz 2023-10-18T19:23:28.109 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90358.log 2023-10-18T19:23:28.110 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130495.log.gz 2023-10-18T19:23:28.110 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104451.log 2023-10-18T19:23:28.110 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130581.log.gz 2023-10-18T19:23:28.111 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.90358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98835.log 2023-10-18T19:23:28.111 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90358.log.gz 2023-10-18T19:23:28.111 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105494.log 2023-10-18T19:23:28.111 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104451.log.gz 2023-10-18T19:23:28.112 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100255.log 2023-10-18T19:23:28.112 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98835.log.gz 2023-10-18T19:23:28.112 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142904.log 2023-10-18T19:23:28.112 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105494.log.gz 2023-10-18T19:23:28.113 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125852.log 2023-10-18T19:23:28.113 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100255.log.gz 2023-10-18T19:23:28.113 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.142904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129149.log 2023-10-18T19:23:28.113 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.142904.log.gz 2023-10-18T19:23:28.114 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125852.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125852.log.gz 2023-10-18T19:23:28.114 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105937.log 2023-10-18T19:23:28.114 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140061.log 2023-10-18T19:23:28.115 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129149.log.gz 2023-10-18T19:23:28.115 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105937.log: gzip -5 0.0%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.105937.log.gz/var/log/ceph/ceph-client.admin.97433.log 2023-10-18T19:23:28.115 INFO:teuthology.orchestra.run.smithi082.stderr: 2023-10-18T19:23:28.115 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.140061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118280.log 2023-10-18T19:23:28.116 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.140061.log.gz 2023-10-18T19:23:28.116 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109750.log 2023-10-18T19:23:28.116 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97433.log.gz 2023-10-18T19:23:28.117 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118280.log: gzip 0.0%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.118280.log.gz-- 2023-10-18T19:23:28.117 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97076.log 2023-10-18T19:23:28.117 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108908.log 2023-10-18T19:23:28.117 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109750.log.gz 2023-10-18T19:23:28.118 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109166.log 2023-10-18T19:23:28.118 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97076.log.gz 2023-10-18T19:23:28.118 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102214.log 2023-10-18T19:23:28.118 INFO:teuthology.orchestra.run.smithi082.stderr: 23.2% -- replaced with /var/log/ceph/ceph-client.admin.108908.log.gz 2023-10-18T19:23:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109166.log: /var/log/ceph/ceph-client.admin.141312.log 2023-10-18T19:23:28.119 INFO:teuthology.orchestra.run.smithi082.stderr: 22.1% -- replaced with /var/log/ceph/ceph-client.admin.109166.log.gz 2023-10-18T19:23:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129893.log 2023-10-18T19:23:28.120 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102214.log.gz 2023-10-18T19:23:28.120 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.141312.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.141312.log.gz 2023-10-18T19:23:28.120 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96991.log 2023-10-18T19:23:28.121 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110223.log 2023-10-18T19:23:28.121 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129893.log.gz 2023-10-18T19:23:28.121 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122725.log 2023-10-18T19:23:28.121 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.96991.log.gz 2023-10-18T19:23:28.122 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110223.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110223.log.gz 2023-10-18T19:23:28.122 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95129.log 2023-10-18T19:23:28.122 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119009.log 2023-10-18T19:23:28.123 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122725.log.gz 2023-10-18T19:23:28.123 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110051.log 2023-10-18T19:23:28.123 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95129.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95129.log.gz 2023-10-18T19:23:28.123 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119009.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119009.log.gz 2023-10-18T19:23:28.124 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96550.log 2023-10-18T19:23:28.124 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96163.log 2023-10-18T19:23:28.124 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110051.log.gz 2023-10-18T19:23:28.124 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135670.log 2023-10-18T19:23:28.125 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96550.log.gz 2023-10-18T19:23:28.125 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102750.log 2023-10-18T19:23:28.125 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96163.log.gz 2023-10-18T19:23:28.126 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134728.log 2023-10-18T19:23:28.126 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135670.log.gz 2023-10-18T19:23:28.126 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101330.log 2023-10-18T19:23:28.126 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102750.log.gz 2023-10-18T19:23:28.127 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95151.log 2023-10-18T19:23:28.127 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134728.log.gz 2023-10-18T19:23:28.127 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95925.log 2023-10-18T19:23:28.127 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101330.log.gz 2023-10-18T19:23:28.128 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92124.log 2023-10-18T19:23:28.128 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95151.log.gz 2023-10-18T19:23:28.128 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101534.log 2023-10-18T19:23:28.129 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95925.log.gz 2023-10-18T19:23:28.129 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119934.log 2023-10-18T19:23:28.129 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.92124.log.gz 2023-10-18T19:23:28.129 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126283.log 2023-10-18T19:23:28.130 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101534.log.gz 2023-10-18T19:23:28.130 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120429.log 2023-10-18T19:23:28.130 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119934.log.gz 2023-10-18T19:23:28.131 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97841.log 2023-10-18T19:23:28.131 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126283.log.gz 2023-10-18T19:23:28.131 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125250.log 2023-10-18T19:23:28.131 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120429.log.gz 2023-10-18T19:23:28.132 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142168.log 2023-10-18T19:23:28.132 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97841.log.gz 2023-10-18T19:23:28.132 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106648.log 2023-10-18T19:23:28.133 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125250.log.gz 2023-10-18T19:23:28.133 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.142168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120322.log 2023-10-18T19:23:28.133 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.142168.log.gz 2023-10-18T19:23:28.133 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97688.log 2023-10-18T19:23:28.134 INFO:teuthology.orchestra.run.smithi082.stderr: 54.0% -- replaced with /var/log/ceph/ceph-client.admin.106648.log.gz 2023-10-18T19:23:28.134 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103317.log 2023-10-18T19:23:28.134 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120322.log.gz 2023-10-18T19:23:28.134 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126205.log 2023-10-18T19:23:28.135 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.97688.log.gz 2023-10-18T19:23:28.135 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102897.log 2023-10-18T19:23:28.135 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103317.log.gz 2023-10-18T19:23:28.135 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132377.log 2023-10-18T19:23:28.136 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126205.log.gz 2023-10-18T19:23:28.136 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111478.log 2023-10-18T19:23:28.136 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102897.log.gz 2023-10-18T19:23:28.137 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114054.log 2023-10-18T19:23:28.137 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132377.log.gz 2023-10-18T19:23:28.137 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133267.log 2023-10-18T19:23:28.137 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111478.log.gz 2023-10-18T19:23:28.138 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131926.log 2023-10-18T19:23:28.138 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114054.log.gz 2023-10-18T19:23:28.138 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102422.log 2023-10-18T19:23:28.139 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133267.log.gz 2023-10-18T19:23:28.139 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101874.log 2023-10-18T19:23:28.139 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131926.log.gz 2023-10-18T19:23:28.139 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95086.log 2023-10-18T19:23:28.140 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102422.log.gz 2023-10-18T19:23:28.140 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129856.log 2023-10-18T19:23:28.140 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101874.log.gz 2023-10-18T19:23:28.141 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115805.log 2023-10-18T19:23:28.141 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95086.log.gz 2023-10-18T19:23:28.141 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121681.log 2023-10-18T19:23:28.141 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129856.log.gz 2023-10-18T19:23:28.142 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93735.log 2023-10-18T19:23:28.142 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115805.log.gz 2023-10-18T19:23:28.142 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140791.log 2023-10-18T19:23:28.143 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121681.log.gz 2023-10-18T19:23:28.143 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93477.log 2023-10-18T19:23:28.143 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93735.log.gz 2023-10-18T19:23:28.143 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.140791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117507.log 2023-10-18T19:23:28.144 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.140791.log.gz 2023-10-18T19:23:28.144 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104989.log 2023-10-18T19:23:28.144 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93477.log.gz 2023-10-18T19:23:28.145 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.117507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98591.log 2023-10-18T19:23:28.145 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117507.log.gz 2023-10-18T19:23:28.145 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129360.log 2023-10-18T19:23:28.145 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104989.log.gz 2023-10-18T19:23:28.146 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140623.log 2023-10-18T19:23:28.146 INFO:teuthology.orchestra.run.smithi082.stderr: 56.5% -- replaced with /var/log/ceph/ceph-client.admin.98591.log.gz 2023-10-18T19:23:28.146 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119195.log 2023-10-18T19:23:28.147 INFO:teuthology.orchestra.run.smithi082.stderr: 53.1% -- replaced with /var/log/ceph/ceph-client.admin.129360.log.gz 2023-10-18T19:23:28.147 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.140623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113648.log 2023-10-18T19:23:28.147 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.140623.log.gz 2023-10-18T19:23:28.147 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128113.log 2023-10-18T19:23:28.148 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119195.log.gz 2023-10-18T19:23:28.148 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96923.log 2023-10-18T19:23:28.148 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113648.log.gz 2023-10-18T19:23:28.149 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135650.log 2023-10-18T19:23:28.149 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128113.log.gz 2023-10-18T19:23:28.149 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125938.log 2023-10-18T19:23:28.150 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.96923.log.gz 2023-10-18T19:23:28.150 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141010.log 2023-10-18T19:23:28.151 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135650.log.gz 2023-10-18T19:23:28.151 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112433.log 2023-10-18T19:23:28.152 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125938.log.gz 2023-10-18T19:23:28.152 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.141010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131146.log 2023-10-18T19:23:28.152 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.141010.log.gz 2023-10-18T19:23:28.153 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103044.log 2023-10-18T19:23:28.153 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.112433.log.gz 2023-10-18T19:23:28.154 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126608.log 2023-10-18T19:23:28.154 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131146.log.gz 2023-10-18T19:23:28.154 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132528.log 2023-10-18T19:23:28.155 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103044.log.gz 2023-10-18T19:23:28.155 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126608.log: gzip 0.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.126608.log.gz 2023-10-18T19:23:28.156 INFO:teuthology.orchestra.run.smithi082.stderr:--verbose -- /var/log/ceph/ceph-client.admin.131904.log 2023-10-18T19:23:28.156 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106751.log 2023-10-18T19:23:28.156 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132528.log.gz 2023-10-18T19:23:28.157 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131904.log: gzip -5 --verbose 0.0%-- /var/log/ceph/ceph-client.admin.115254.log -- replaced with /var/log/ceph/ceph-client.admin.131904.log.gz 2023-10-18T19:23:28.157 INFO:teuthology.orchestra.run.smithi082.stderr: 2023-10-18T19:23:28.157 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115676.log 2023-10-18T19:23:28.158 INFO:teuthology.orchestra.run.smithi082.stderr: 53.2% -- replaced with /var/log/ceph/ceph-client.admin.106751.log.gz 2023-10-18T19:23:28.158 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128150.log 2023-10-18T19:23:28.158 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115254.log.gz 2023-10-18T19:23:28.158 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122811.log 2023-10-18T19:23:28.159 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115676.log.gz 2023-10-18T19:23:28.159 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91198.log 2023-10-18T19:23:28.159 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128150.log.gz 2023-10-18T19:23:28.159 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120515.log 2023-10-18T19:23:28.160 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122811.log.gz 2023-10-18T19:23:28.160 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119377.log 2023-10-18T19:23:28.160 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91198.log.gz 2023-10-18T19:23:28.161 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100335.log 2023-10-18T19:23:28.161 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120515.log.gz 2023-10-18T19:23:28.161 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93585.log 2023-10-18T19:23:28.161 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119377.log.gz 2023-10-18T19:23:28.162 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126554.log 2023-10-18T19:23:28.162 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100335.log.gz 2023-10-18T19:23:28.162 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94570.log 2023-10-18T19:23:28.162 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93585.log.gz 2023-10-18T19:23:28.163 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100803.log 2023-10-18T19:23:28.163 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126554.log.gz 2023-10-18T19:23:28.163 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131390.log 2023-10-18T19:23:28.164 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94570.log.gz 2023-10-18T19:23:28.164 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124734.log 2023-10-18T19:23:28.164 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100803.log.gz 2023-10-18T19:23:28.164 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115337.log 2023-10-18T19:23:28.165 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131390.log.gz 2023-10-18T19:23:28.165 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124734.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.124734.log.gz 2023-10-18T19:23:28.165 INFO:teuthology.orchestra.run.smithi082.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.122197.log 2023-10-18T19:23:28.165 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91614.log 2023-10-18T19:23:28.166 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115337.log.gz 2023-10-18T19:23:28.166 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93872.log 2023-10-18T19:23:28.166 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122197.log.gz 2023-10-18T19:23:28.167 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133411.log 2023-10-18T19:23:28.167 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91614.log.gz 2023-10-18T19:23:28.167 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121131.log 2023-10-18T19:23:28.167 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93872.log.gz 2023-10-18T19:23:28.168 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96850.log 2023-10-18T19:23:28.168 INFO:teuthology.orchestra.run.smithi082.stderr: 12.0% -- replaced with /var/log/ceph/ceph-client.admin.133411.log.gz 2023-10-18T19:23:28.168 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121131.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121131.log.gz 2023-10-18T19:23:28.168 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99175.log 2023-10-18T19:23:28.169 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102632.log 2023-10-18T19:23:28.169 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96850.log.gz 2023-10-18T19:23:28.169 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94656.log 2023-10-18T19:23:28.170 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99175.log.gz 2023-10-18T19:23:28.170 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101857.log 2023-10-18T19:23:28.170 INFO:teuthology.orchestra.run.smithi082.stderr: 22.8% -- replaced with /var/log/ceph/ceph-client.admin.102632.log.gz 2023-10-18T19:23:28.170 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124691.log 2023-10-18T19:23:28.171 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94656.log.gz 2023-10-18T19:23:28.171 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142442.log 2023-10-18T19:23:28.171 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101857.log.gz 2023-10-18T19:23:28.171 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124691.log: gzip 0.0%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.124691.log.gz-- 2023-10-18T19:23:28.172 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.117743.log 2023-10-18T19:23:28.172 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.142442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93714.log 2023-10-18T19:23:28.172 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.142442.log.gz 2023-10-18T19:23:28.173 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.117743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115783.log 2023-10-18T19:23:28.173 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117743.log.gz 2023-10-18T19:23:28.173 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125314.log 2023-10-18T19:23:28.173 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93714.log.gz 2023-10-18T19:23:28.174 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140145.log 2023-10-18T19:23:28.174 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115783.log.gz 2023-10-18T19:23:28.177 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95538.log 2023-10-18T19:23:28.177 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125314.log.gz 2023-10-18T19:23:28.177 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.140145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113236.log 2023-10-18T19:23:28.178 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.140145.log.gz 2023-10-18T19:23:28.178 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130918.log 2023-10-18T19:23:28.178 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95538.log.gz 2023-10-18T19:23:28.179 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134962.log 2023-10-18T19:23:28.179 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113236.log.gz 2023-10-18T19:23:28.179 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108626.log 2023-10-18T19:23:28.179 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130918.log.gz 2023-10-18T19:23:28.180 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135284.log 2023-10-18T19:23:28.180 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134962.log.gz 2023-10-18T19:23:28.180 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126785.log 2023-10-18T19:23:28.181 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108626.log.gz 2023-10-18T19:23:28.181 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122175.log 2023-10-18T19:23:28.181 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135284.log.gz 2023-10-18T19:23:28.181 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92286.log 2023-10-18T19:23:28.182 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126785.log.gz 2023-10-18T19:23:28.182 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115654.log 2023-10-18T19:23:28.182 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122175.log.gz 2023-10-18T19:23:28.182 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119697.log 2023-10-18T19:23:28.183 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92286.log.gz 2023-10-18T19:23:28.183 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97892.log 2023-10-18T19:23:28.183 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115654.log.gz 2023-10-18T19:23:28.183 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139365.log 2023-10-18T19:23:28.184 INFO:teuthology.orchestra.run.smithi082.stderr: 22.4% -- replaced with /var/log/ceph/ceph-client.admin.119697.log.gz 2023-10-18T19:23:28.184 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98470.log 2023-10-18T19:23:28.184 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97892.log.gz 2023-10-18T19:23:28.185 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.139365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125830.log 2023-10-18T19:23:28.185 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.139365.log.gz 2023-10-18T19:23:28.185 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93520.log 2023-10-18T19:23:28.185 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.98470.log.gz 2023-10-18T19:23:28.186 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125830.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125830.log.gz 2023-10-18T19:23:28.186 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103758.log 2023-10-18T19:23:28.186 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116927.log 2023-10-18T19:23:28.187 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93520.log.gz 2023-10-18T19:23:28.187 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107133.log 2023-10-18T19:23:28.188 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103758.log.gz 2023-10-18T19:23:28.188 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.116927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132184.log 2023-10-18T19:23:28.189 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116927.log.gz 2023-10-18T19:23:28.189 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117979.log 2023-10-18T19:23:28.190 INFO:teuthology.orchestra.run.smithi082.stderr: 54.3% -- replaced with /var/log/ceph/ceph-client.admin.107133.log.gz 2023-10-18T19:23:28.190 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127661.log 2023-10-18T19:23:28.191 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132184.log.gz 2023-10-18T19:23:28.191 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.117979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131186.log 2023-10-18T19:23:28.192 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117979.log.gz 2023-10-18T19:23:28.192 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127661.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.127661.log.gz-5 2023-10-18T19:23:28.193 INFO:teuthology.orchestra.run.smithi082.stderr:--verbose -- /var/log/ceph/ceph-client.admin.96572.log 2023-10-18T19:23:28.193 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132205.log 2023-10-18T19:23:28.194 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131186.log.gz 2023-10-18T19:23:28.194 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134564.log 2023-10-18T19:23:28.194 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96572.log.gz 2023-10-18T19:23:28.195 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132205.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132205.log.gz 2023-10-18T19:23:28.195 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94033.log 2023-10-18T19:23:28.195 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97212.log 2023-10-18T19:23:28.196 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134564.log.gz 2023-10-18T19:23:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116256.log 2023-10-18T19:23:28.196 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94033.log.gz 2023-10-18T19:23:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105916.log 2023-10-18T19:23:28.197 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.97212.log.gz 2023-10-18T19:23:28.197 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.116256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125680.log 2023-10-18T19:23:28.197 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116256.log.gz 2023-10-18T19:23:28.198 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112062.log 2023-10-18T19:23:28.198 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105916.log.gz 2023-10-18T19:23:28.198 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92983.log 2023-10-18T19:23:28.198 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125680.log.gz 2023-10-18T19:23:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121982.log 2023-10-18T19:23:28.199 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112062.log.gz 2023-10-18T19:23:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114550.log 2023-10-18T19:23:28.200 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92983.log.gz 2023-10-18T19:23:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139851.log 2023-10-18T19:23:28.200 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121982.log.gz 2023-10-18T19:23:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114550.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114550.log.gz 2023-10-18T19:23:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91270.log 2023-10-18T19:23:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.139851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104199.log 2023-10-18T19:23:28.201 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.139851.log.gz 2023-10-18T19:23:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111420.log 2023-10-18T19:23:28.202 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91270.log.gz 2023-10-18T19:23:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124863.log 2023-10-18T19:23:28.203 INFO:teuthology.orchestra.run.smithi082.stderr: 23.7% -- replaced with /var/log/ceph/ceph-client.admin.104199.log.gz 2023-10-18T19:23:28.203 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133369.log 2023-10-18T19:23:28.203 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.111420.log.gz 2023-10-18T19:23:28.203 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95657.log 2023-10-18T19:23:28.204 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124863.log.gz 2023-10-18T19:23:28.204 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127289.log 2023-10-18T19:23:28.204 INFO:teuthology.orchestra.run.smithi082.stderr: 11.1% -- replaced with /var/log/ceph/ceph-client.admin.133369.log.gz 2023-10-18T19:23:28.204 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120451.log 2023-10-18T19:23:28.205 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95657.log.gz 2023-10-18T19:23:28.205 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103884.log 2023-10-18T19:23:28.205 INFO:teuthology.orchestra.run.smithi082.stderr: 53.1% -- replaced with /var/log/ceph/ceph-client.admin.127289.log.gz 2023-10-18T19:23:28.205 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128394.log 2023-10-18T19:23:28.206 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120451.log.gz 2023-10-18T19:23:28.206 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94356.log 2023-10-18T19:23:28.206 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103884.log.gz 2023-10-18T19:23:28.207 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125508.log 2023-10-18T19:23:28.207 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128394.log.gz 2023-10-18T19:23:28.207 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135177.log 2023-10-18T19:23:28.208 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94356.log.gz 2023-10-18T19:23:28.208 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104409.log 2023-10-18T19:23:28.208 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125508.log.gz 2023-10-18T19:23:28.208 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134876.log 2023-10-18T19:23:28.209 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135177.log.gz 2023-10-18T19:23:28.209 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143038.log 2023-10-18T19:23:28.209 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104409.log.gz 2023-10-18T19:23:28.209 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141354.log 2023-10-18T19:23:28.210 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134876.log.gz 2023-10-18T19:23:28.210 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.143038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130796.log 2023-10-18T19:23:28.210 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.143038.log.gz 2023-10-18T19:23:28.211 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.141354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98572.log 2023-10-18T19:23:28.211 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.141354.log.gz 2023-10-18T19:23:28.211 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131539.log 2023-10-18T19:23:28.211 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130796.log.gz 2023-10-18T19:23:28.212 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124002.log 2023-10-18T19:23:28.212 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98572.log.gz 2023-10-18T19:23:28.212 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130237.log 2023-10-18T19:23:28.213 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131539.log.gz 2023-10-18T19:23:28.213 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90610.log 2023-10-18T19:23:28.214 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124002.log.gz 2023-10-18T19:23:28.214 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121361.log 2023-10-18T19:23:28.214 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130237.log.gz 2023-10-18T19:23:28.215 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.90610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135220.log 2023-10-18T19:23:28.215 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90610.log.gz 2023-10-18T19:23:28.215 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103191.log 2023-10-18T19:23:28.215 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121361.log.gz 2023-10-18T19:23:28.216 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101262.log 2023-10-18T19:23:28.216 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135220.log.gz 2023-10-18T19:23:28.216 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130559.log 2023-10-18T19:23:28.217 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103191.log.gz 2023-10-18T19:23:28.217 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110420.log 2023-10-18T19:23:28.217 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101262.log.gz 2023-10-18T19:23:28.218 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111070.log 2023-10-18T19:23:28.218 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130559.log.gz 2023-10-18T19:23:28.219 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94507.log 2023-10-18T19:23:28.219 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110420.log.gz 2023-10-18T19:23:28.220 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120709.log 2023-10-18T19:23:28.220 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111070.log.gz 2023-10-18T19:23:28.221 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131496.log 2023-10-18T19:23:28.221 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94507.log.gz 2023-10-18T19:23:28.221 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93305.log 2023-10-18T19:23:28.222 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120709.log.gz 2023-10-18T19:23:28.222 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141874.log 2023-10-18T19:23:28.223 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131496.log.gz 2023-10-18T19:23:28.223 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115398.log 2023-10-18T19:23:28.224 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93305.log.gz 2023-10-18T19:23:28.224 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.141874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123176.log 2023-10-18T19:23:28.225 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.141874.log.gz 2023-10-18T19:23:28.225 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137856.log 2023-10-18T19:23:28.226 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115398.log.gz 2023-10-18T19:23:28.226 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95754.log 2023-10-18T19:23:28.227 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123176.log.gz 2023-10-18T19:23:28.227 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.137856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112636.log 2023-10-18T19:23:28.228 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.137856.log.gz 2023-10-18T19:23:28.228 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96421.log 2023-10-18T19:23:28.229 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95754.log.gz 2023-10-18T19:23:28.229 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112191.log 2023-10-18T19:23:28.230 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112636.log.gz 2023-10-18T19:23:28.231 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106117.log 2023-10-18T19:23:28.231 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96421.log.gz 2023-10-18T19:23:28.232 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107565.log 2023-10-18T19:23:28.232 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112191.log.gz 2023-10-18T19:23:28.233 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123262.log 2023-10-18T19:23:28.233 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106117.log.gz 2023-10-18T19:23:28.234 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131883.log 2023-10-18T19:23:28.234 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107565.log.gz 2023-10-18T19:23:28.235 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106565.log 2023-10-18T19:23:28.235 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123262.log.gz 2023-10-18T19:23:28.235 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90864.log 2023-10-18T19:23:28.236 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131883.log.gz 2023-10-18T19:23:28.236 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117807.log 2023-10-18T19:23:28.237 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106565.log.gz 2023-10-18T19:23:28.237 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.90864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122458.log 2023-10-18T19:23:28.237 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90864.log.gz 2023-10-18T19:23:28.238 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.117807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117786.log 2023-10-18T19:23:28.238 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117807.log.gz 2023-10-18T19:23:28.239 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97144.log 2023-10-18T19:23:28.239 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122458.log.gz 2023-10-18T19:23:28.240 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.117786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120945.log 2023-10-18T19:23:28.240 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117786.log.gz 2023-10-18T19:23:28.240 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121595.log 2023-10-18T19:23:28.240 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97144.log.gz 2023-10-18T19:23:28.241 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91549.log 2023-10-18T19:23:28.241 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120945.log.gz 2023-10-18T19:23:28.241 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110742.log 2023-10-18T19:23:28.241 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121595.log.gz 2023-10-18T19:23:28.242 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115955.log 2023-10-18T19:23:28.242 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91549.log.gz 2023-10-18T19:23:28.243 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102335.log 2023-10-18T19:23:28.243 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110742.log.gz 2023-10-18T19:23:28.243 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133160.log 2023-10-18T19:23:28.244 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115955.log.gz 2023-10-18T19:23:28.244 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116235.log 2023-10-18T19:23:28.245 INFO:teuthology.orchestra.run.smithi082.stderr: 52.7% -- replaced with /var/log/ceph/ceph-client.admin.102335.log.gz 2023-10-18T19:23:28.245 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125056.log 2023-10-18T19:23:28.245 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133160.log.gz 2023-10-18T19:23:28.246 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.116235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100035.log 2023-10-18T19:23:28.246 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116235.log.gz 2023-10-18T19:23:28.246 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97705.log 2023-10-18T19:23:28.247 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125056.log.gz 2023-10-18T19:23:28.247 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131754.log 2023-10-18T19:23:28.247 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100035.log.gz 2023-10-18T19:23:28.247 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91420.log 2023-10-18T19:23:28.248 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97705.log.gz 2023-10-18T19:23:28.248 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122577.log 2023-10-18T19:23:28.248 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131754.log.gz 2023-10-18T19:23:28.248 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128474.log 2023-10-18T19:23:28.249 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91420.log.gz 2023-10-18T19:23:28.249 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102265.log 2023-10-18T19:23:28.249 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122577.log.gz 2023-10-18T19:23:28.250 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110094.log 2023-10-18T19:23:28.250 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128474.log.gz 2023-10-18T19:23:28.250 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133673.log 2023-10-18T19:23:28.251 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102265.log.gz 2023-10-18T19:23:28.251 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110094.log: 0.0%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.110094.log.gz--verbose 2023-10-18T19:23:28.251 INFO:teuthology.orchestra.run.smithi082.stderr:-- /var/log/ceph/ceph-client.admin.111121.log 2023-10-18T19:23:28.251 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100515.log 2023-10-18T19:23:28.252 INFO:teuthology.orchestra.run.smithi082.stderr: 26.8% -- replaced with /var/log/ceph/ceph-client.admin.133673.log.gz 2023-10-18T19:23:28.252 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117872.log 2023-10-18T19:23:28.252 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111121.log.gz 2023-10-18T19:23:28.253 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100515.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100515.log.gzgzip 2023-10-18T19:23:28.253 INFO:teuthology.orchestra.run.smithi082.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.91807.log 2023-10-18T19:23:28.253 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.117872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138292.log 2023-10-18T19:23:28.253 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117872.log.gz 2023-10-18T19:23:28.254 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96615.log 2023-10-18T19:23:28.254 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91807.log.gz 2023-10-18T19:23:28.254 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.138292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91506.log 2023-10-18T19:23:28.255 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.138292.log.gz 2023-10-18T19:23:28.255 INFO:teuthology.orchestra.run.smithi082.stderr:gzip /var/log/ceph/ceph-client.admin.96615.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.92666.log 2023-10-18T19:23:28.255 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96615.log.gz 2023-10-18T19:23:28.255 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101993.log 2023-10-18T19:23:28.256 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91506.log.gz 2023-10-18T19:23:28.256 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120881.log 2023-10-18T19:23:28.256 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92666.log.gz 2023-10-18T19:23:28.256 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113176.log 2023-10-18T19:23:28.257 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101993.log.gz 2023-10-18T19:23:28.257 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126495.log 2023-10-18T19:23:28.257 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120881.log.gz 2023-10-18T19:23:28.258 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113176.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.113176.log.gz-5 2023-10-18T19:23:28.258 INFO:teuthology.orchestra.run.smithi082.stderr:--verbose -- /var/log/ceph/ceph-client.admin.114011.log 2023-10-18T19:23:28.258 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123984.log 2023-10-18T19:23:28.258 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126495.log.gz 2023-10-18T19:23:28.259 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132872.log 2023-10-18T19:23:28.259 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114011.log.gz 2023-10-18T19:23:28.259 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110180.log 2023-10-18T19:23:28.260 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123984.log.gz 2023-10-18T19:23:28.260 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106834.log 2023-10-18T19:23:28.260 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132872.log.gz 2023-10-18T19:23:28.260 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119356.log 2023-10-18T19:23:28.261 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110180.log.gz 2023-10-18T19:23:28.261 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106834.log: gzip 0.0%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.106834.log.gz-- 2023-10-18T19:23:28.261 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123587.log 2023-10-18T19:23:28.262 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126357.log 2023-10-18T19:23:28.262 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119356.log.gz 2023-10-18T19:23:28.262 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128747.log 2023-10-18T19:23:28.262 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123587.log.gz 2023-10-18T19:23:28.263 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126357.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126357.log.gzgzip 2023-10-18T19:23:28.263 INFO:teuthology.orchestra.run.smithi082.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.101194.log 2023-10-18T19:23:28.263 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120257.log 2023-10-18T19:23:28.264 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128747.log.gz 2023-10-18T19:23:28.264 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127859.log 2023-10-18T19:23:28.264 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101194.log.gz 2023-10-18T19:23:28.264 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104178.log 2023-10-18T19:23:28.265 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120257.log.gz 2023-10-18T19:23:28.265 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107876.log 2023-10-18T19:23:28.265 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127859.log.gz 2023-10-18T19:23:28.265 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117678.log 2023-10-18T19:23:28.266 INFO:teuthology.orchestra.run.smithi082.stderr: 23.2% -- replaced with /var/log/ceph/ceph-client.admin.104178.log.gz 2023-10-18T19:23:28.266 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107876.log: gzip 0.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.107876.log.gz--verbose 2023-10-18T19:23:28.266 INFO:teuthology.orchestra.run.smithi082.stderr:-- /var/log/ceph/ceph-client.admin.91743.log 2023-10-18T19:23:28.267 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.117678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119316.log 2023-10-18T19:23:28.267 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117678.log.gz 2023-10-18T19:23:28.267 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135349.log 2023-10-18T19:23:28.268 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91743.log.gz 2023-10-18T19:23:28.268 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96271.log 2023-10-18T19:23:28.269 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119316.log.gz 2023-10-18T19:23:28.269 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122658.log 2023-10-18T19:23:28.270 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135349.log.gz 2023-10-18T19:23:28.270 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96271.log: /var/log/ceph/ceph-client.admin.133180.log 2023-10-18T19:23:28.271 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96271.log.gz 2023-10-18T19:23:28.271 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122658.log: 0.0%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.122658.log.gz--verbose 2023-10-18T19:23:28.272 INFO:teuthology.orchestra.run.smithi082.stderr:-- /var/log/ceph/ceph-client.admin.104346.log 2023-10-18T19:23:28.272 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110721.log 2023-10-18T19:23:28.273 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133180.log.gz 2023-10-18T19:23:28.273 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119719.log 2023-10-18T19:23:28.274 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104346.log.gz 2023-10-18T19:23:28.274 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101602.log 2023-10-18T19:23:28.275 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110721.log.gz 2023-10-18T19:23:28.275 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96056.log 2023-10-18T19:23:28.275 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119719.log.gz 2023-10-18T19:23:28.276 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110699.log 2023-10-18T19:23:28.276 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101602.log.gz 2023-10-18T19:23:28.276 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96056.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96056.log.gz 2023-10-18T19:23:28.276 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113669.log 2023-10-18T19:23:28.277 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92418.log 2023-10-18T19:23:28.277 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110699.log.gz 2023-10-18T19:23:28.277 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127521.log 2023-10-18T19:23:28.277 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113669.log.gz 2023-10-18T19:23:28.278 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113733.log 2023-10-18T19:23:28.278 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.92418.log.gz 2023-10-18T19:23:28.278 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101636.log 2023-10-18T19:23:28.278 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127521.log.gz 2023-10-18T19:23:28.279 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90822.log 2023-10-18T19:23:28.279 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113733.log.gz 2023-10-18T19:23:28.279 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107919.log 2023-10-18T19:23:28.280 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101636.log.gz 2023-10-18T19:23:28.280 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.90822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131840.log 2023-10-18T19:23:28.280 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90822.log.gz 2023-10-18T19:23:28.280 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114291.log 2023-10-18T19:23:28.281 INFO:teuthology.orchestra.run.smithi082.stderr: 52.8% -- replaced with /var/log/ceph/ceph-client.admin.107919.log.gz 2023-10-18T19:23:28.281 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131840.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131840.log.gz 2023-10-18T19:23:28.281 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107981.log 2023-10-18T19:23:28.281 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127249.log 2023-10-18T19:23:28.282 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114291.log.gz 2023-10-18T19:23:28.282 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104709.log 2023-10-18T19:23:28.282 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107981.log.gz 2023-10-18T19:23:28.283 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90948.log 2023-10-18T19:23:28.283 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127249.log.gz 2023-10-18T19:23:28.283 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138987.log 2023-10-18T19:23:28.283 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104709.log.gz 2023-10-18T19:23:28.284 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.90948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129069.log 2023-10-18T19:23:28.284 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90948.log.gz 2023-10-18T19:23:28.284 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.138987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130108.log 2023-10-18T19:23:28.284 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.138987.log.gz 2023-10-18T19:23:28.285 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94464.log 2023-10-18T19:23:28.285 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129069.log.gz 2023-10-18T19:23:28.285 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96120.log 2023-10-18T19:23:28.285 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130108.log.gz 2023-10-18T19:23:28.286 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118151.log 2023-10-18T19:23:28.286 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94464.log.gz 2023-10-18T19:23:28.286 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96120.log: gzip 0.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.96120.log.gz--verbose 2023-10-18T19:23:28.287 INFO:teuthology.orchestra.run.smithi082.stderr:-- /var/log/ceph/ceph-client.admin.129437.log 2023-10-18T19:23:28.287 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124159.log 2023-10-18T19:23:28.287 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118151.log.gz 2023-10-18T19:23:28.287 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93953.log 2023-10-18T19:23:28.288 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129437.log.gz 2023-10-18T19:23:28.288 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124159.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124159.log.gz 2023-10-18T19:23:28.288 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98855.log 2023-10-18T19:23:28.288 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93991.log 2023-10-18T19:23:28.289 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93953.log.gz 2023-10-18T19:23:28.289 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125185.log 2023-10-18T19:23:28.289 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98855.log.gz 2023-10-18T19:23:28.290 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93991.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.93991.log.gz-5 2023-10-18T19:23:28.290 INFO:teuthology.orchestra.run.smithi082.stderr:--verbose -- /var/log/ceph/ceph-client.admin.92243.log 2023-10-18T19:23:28.290 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104946.log 2023-10-18T19:23:28.290 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125185.log.gz 2023-10-18T19:23:28.291 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96906.log 2023-10-18T19:23:28.291 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92243.log.gz 2023-10-18T19:23:28.291 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104946.log: gzip -5 --verbose -- 0.0%/var/log/ceph/ceph-client.admin.112594.log 2023-10-18T19:23:28.291 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.104946.log.gz 2023-10-18T19:23:28.292 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142126.log 2023-10-18T19:23:28.292 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.96906.log.gz 2023-10-18T19:23:28.292 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119296.log 2023-10-18T19:23:28.293 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112594.log.gz 2023-10-18T19:23:28.293 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.142126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131019.log 2023-10-18T19:23:28.293 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.142126.log.gz 2023-10-18T19:23:28.293 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97331.log 2023-10-18T19:23:28.294 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119296.log.gz 2023-10-18T19:23:28.294 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104136.log 2023-10-18T19:23:28.294 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131019.log.gz 2023-10-18T19:23:28.295 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111064.log 2023-10-18T19:23:28.295 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97331.log.gz 2023-10-18T19:23:28.295 INFO:teuthology.orchestra.run.smithi082.stderr:gzip /var/log/ceph/ceph-client.admin.104136.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.106418.log 2023-10-18T19:23:28.295 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104136.log.gz 2023-10-18T19:23:28.296 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97654.log 2023-10-18T19:23:28.296 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111064.log.gz 2023-10-18T19:23:28.296 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106418.log: gzip -5 0.0%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.106418.log.gz 2023-10-18T19:23:28.296 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115419.log 2023-10-18T19:23:28.297 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131206.log 2023-10-18T19:23:28.297 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97654.log.gz 2023-10-18T19:23:28.297 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122003.log 2023-10-18T19:23:28.298 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115419.log.gz 2023-10-18T19:23:28.298 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131206.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-18T19:23:28.298 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131206.log.gz 2023-10-18T19:23:28.298 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95344.log 2023-10-18T19:23:28.299 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122003.log.gz 2023-10-18T19:23:28.299 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101517.log 2023-10-18T19:23:28.299 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95344.log: gzip -5 0.0%--verbose -- replaced with /var/log/ceph/ceph-client.admin.95344.log.gz-- 2023-10-18T19:23:28.299 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.116987.log 2023-10-18T19:23:28.300 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107312.log 2023-10-18T19:23:28.300 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101517.log.gz 2023-10-18T19:23:28.300 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.116987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105225.log 2023-10-18T19:23:28.300 INFO:teuthology.orchestra.run.smithi082.stderr: 55.1% -- replaced with /var/log/ceph/ceph-client.admin.116987.log.gz 2023-10-18T19:23:28.301 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129794.log 2023-10-18T19:23:28.301 INFO:teuthology.orchestra.run.smithi082.stderr: 53.2% -- replaced with /var/log/ceph/ceph-client.admin.107312.log.gz 2023-10-18T19:23:28.301 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126826.log 2023-10-18T19:23:28.301 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105225.log.gz 2023-10-18T19:23:28.302 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94850.log 2023-10-18T19:23:28.302 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129794.log.gz 2023-10-18T19:23:28.302 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139977.log 2023-10-18T19:23:28.303 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126826.log.gz 2023-10-18T19:23:28.303 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108288.log 2023-10-18T19:23:28.303 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94850.log.gz 2023-10-18T19:23:28.303 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.139977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95108.log 2023-10-18T19:23:28.304 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.139977.log.gz 2023-10-18T19:23:28.304 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96940.log 2023-10-18T19:23:28.304 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108288.log.gz 2023-10-18T19:23:28.304 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112493.log 2023-10-18T19:23:28.305 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95108.log.gz 2023-10-18T19:23:28.305 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122597.log 2023-10-18T19:23:28.305 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.96940.log.gz 2023-10-18T19:23:28.305 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126571.log 2023-10-18T19:23:28.306 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112493.log.gz 2023-10-18T19:23:28.306 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132291.log 2023-10-18T19:23:28.306 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122597.log.gz 2023-10-18T19:23:28.307 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97518.log 2023-10-18T19:23:28.307 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126571.log.gz 2023-10-18T19:23:28.307 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128331.log 2023-10-18T19:23:28.307 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132291.log.gz 2023-10-18T19:23:28.308 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119998.log 2023-10-18T19:23:28.308 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97518.log.gz 2023-10-18T19:23:28.308 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102061.log 2023-10-18T19:23:28.308 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128331.log.gz 2023-10-18T19:23:28.309 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103926.log 2023-10-18T19:23:28.309 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119998.log.gz 2023-10-18T19:23:28.309 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123047.log 2023-10-18T19:23:28.309 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102061.log.gz 2023-10-18T19:23:28.310 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122768.log 2023-10-18T19:23:28.310 INFO:teuthology.orchestra.run.smithi082.stderr: 23.3% -- replaced with /var/log/ceph/ceph-client.admin.103926.log.gz 2023-10-18T19:23:28.310 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122699.log 2023-10-18T19:23:28.311 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123047.log.gz 2023-10-18T19:23:28.311 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101959.log 2023-10-18T19:23:28.311 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122768.log.gz 2023-10-18T19:23:28.311 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101585.log 2023-10-18T19:23:28.312 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122699.log.gz 2023-10-18T19:23:28.312 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101296.log 2023-10-18T19:23:28.312 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101959.log.gz 2023-10-18T19:23:28.312 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100595.log 2023-10-18T19:23:28.313 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101585.log.gz 2023-10-18T19:23:28.313 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125723.log 2023-10-18T19:23:28.313 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101296.log.gz 2023-10-18T19:23:28.314 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100633.log 2023-10-18T19:23:28.314 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100595.log.gz 2023-10-18T19:23:28.314 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118173.log 2023-10-18T19:23:28.314 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125723.log.gz 2023-10-18T19:23:28.315 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104031.log 2023-10-18T19:23:28.315 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100633.log.gz 2023-10-18T19:23:28.315 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112170.log 2023-10-18T19:23:28.315 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118173.log.gz 2023-10-18T19:23:28.316 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128073.log 2023-10-18T19:23:28.316 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104031.log.gz 2023-10-18T19:23:28.316 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93899.log 2023-10-18T19:23:28.316 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112170.log.gz 2023-10-18T19:23:28.317 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103422.log 2023-10-18T19:23:28.317 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128073.log.gz 2023-10-18T19:23:28.317 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93899.log: gzip -5 --verbose -- 0.0%/var/log/ceph/ceph-client.admin.129265.log 2023-10-18T19:23:28.317 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.93899.log.gz 2023-10-18T19:23:28.318 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132012.log 2023-10-18T19:23:28.318 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103422.log.gz 2023-10-18T19:23:28.318 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115611.log 2023-10-18T19:23:28.319 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129265.log.gz 2023-10-18T19:23:28.319 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130860.log 2023-10-18T19:23:28.319 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132012.log.gz 2023-10-18T19:23:28.319 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132420.log 2023-10-18T19:23:28.320 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115611.log.gz 2023-10-18T19:23:28.320 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139541.log 2023-10-18T19:23:28.320 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130860.log.gz 2023-10-18T19:23:28.320 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103989.log 2023-10-18T19:23:28.321 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132420.log.gz 2023-10-18T19:23:28.321 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.139541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121487.log 2023-10-18T19:23:28.321 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.139541.log.gz 2023-10-18T19:23:28.321 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135327.log 2023-10-18T19:23:28.322 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103989.log.gz 2023-10-18T19:23:28.322 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125465.log 2023-10-18T19:23:28.322 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121487.log.gz 2023-10-18T19:23:28.323 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127841.log 2023-10-18T19:23:28.323 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135327.log.gz 2023-10-18T19:23:28.323 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99615.log 2023-10-18T19:23:28.323 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125465.log.gz 2023-10-18T19:23:28.324 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121724.log 2023-10-18T19:23:28.324 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127841.log.gz 2023-10-18T19:23:28.324 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124196.log 2023-10-18T19:23:28.324 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99615.log.gz 2023-10-18T19:23:28.325 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139767.log 2023-10-18T19:23:28.325 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121724.log.gz 2023-10-18T19:23:28.325 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108127.log 2023-10-18T19:23:28.325 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124196.log.gz 2023-10-18T19:23:28.326 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.139767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119049.log 2023-10-18T19:23:28.326 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.139767.log.gz 2023-10-18T19:23:28.326 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115633.log 2023-10-18T19:23:28.326 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108127.log.gz 2023-10-18T19:23:28.327 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110398.log 2023-10-18T19:23:28.327 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119049.log.gz 2023-10-18T19:23:28.327 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127187.log 2023-10-18T19:23:28.328 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115633.log.gz 2023-10-18T19:23:28.328 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138418.log 2023-10-18T19:23:28.328 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110398.log.gz 2023-10-18T19:23:28.328 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125271.log 2023-10-18T19:23:28.329 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127187.log.gz 2023-10-18T19:23:28.329 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.138418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115912.log 2023-10-18T19:23:28.329 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.138418.log.gz 2023-10-18T19:23:28.329 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123133.log 2023-10-18T19:23:28.330 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125271.log.gz 2023-10-18T19:23:28.330 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106606.log 2023-10-18T19:23:28.330 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115912.log.gz 2023-10-18T19:23:28.330 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122438.log 2023-10-18T19:23:28.331 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123133.log.gz 2023-10-18T19:23:28.331 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104645.log 2023-10-18T19:23:28.331 INFO:teuthology.orchestra.run.smithi082.stderr: 56.5% -- replaced with /var/log/ceph/ceph-client.admin.106606.log.gz 2023-10-18T19:23:28.332 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118022.log 2023-10-18T19:23:28.332 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122438.log.gz 2023-10-18T19:23:28.332 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99555.log 2023-10-18T19:23:28.332 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104645.log.gz 2023-10-18T19:23:28.333 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118022.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118022.log.gz 2023-10-18T19:23:28.333 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125701.log 2023-10-18T19:23:28.333 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99555.log.gz 2023-10-18T19:23:28.333 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104115.log 2023-10-18T19:23:28.334 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104010.log 2023-10-18T19:23:28.334 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125701.log.gz 2023-10-18T19:23:28.334 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125013.log 2023-10-18T19:23:28.335 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104115.log.gz 2023-10-18T19:23:28.335 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131969.log 2023-10-18T19:23:28.335 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104010.log.gz 2023-10-18T19:23:28.335 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103527.log 2023-10-18T19:23:28.336 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125013.log.gz 2023-10-18T19:23:28.336 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99875.log 2023-10-18T19:23:28.336 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131969.log.gz 2023-10-18T19:23:28.336 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124992.log 2023-10-18T19:23:28.337 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103527.log.gz 2023-10-18T19:23:28.337 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91915.log 2023-10-18T19:23:28.337 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99875.log.gz 2023-10-18T19:23:28.337 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134178.log 2023-10-18T19:23:28.338 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124992.log.gz 2023-10-18T19:23:28.338 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91958.log 2023-10-18T19:23:28.338 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91915.log.gz 2023-10-18T19:23:28.338 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132055.log 2023-10-18T19:23:28.339 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134178.log.gz 2023-10-18T19:23:28.339 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133140.log 2023-10-18T19:23:28.339 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91958.log.gz 2023-10-18T19:23:28.340 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135413.log 2023-10-18T19:23:28.340 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132055.log.gz 2023-10-18T19:23:28.340 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106354.log 2023-10-18T19:23:28.340 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133140.log.gz 2023-10-18T19:23:28.341 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99475.log 2023-10-18T19:23:28.341 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135413.log.gz 2023-10-18T19:23:28.341 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107834.log 2023-10-18T19:23:28.341 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106354.log.gz 2023-10-18T19:23:28.342 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141656.log 2023-10-18T19:23:28.342 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99475.log.gz 2023-10-18T19:23:28.342 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120558.log 2023-10-18T19:23:28.342 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107834.log.gz 2023-10-18T19:23:28.343 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.141656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92342.log 2023-10-18T19:23:28.343 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.141656.log.gz 2023-10-18T19:23:28.343 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109814.log 2023-10-18T19:23:28.344 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120558.log.gz 2023-10-18T19:23:28.344 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101942.log 2023-10-18T19:23:28.344 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.92342.log.gz 2023-10-18T19:23:28.344 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105701.log 2023-10-18T19:23:28.345 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109814.log.gz 2023-10-18T19:23:28.345 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101942.log: gzip 0.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.101942.log.gz--verbose 2023-10-18T19:23:28.345 INFO:teuthology.orchestra.run.smithi082.stderr:-- /var/log/ceph/ceph-client.admin.93047.log 2023-10-18T19:23:28.345 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100015.log 2023-10-18T19:23:28.346 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105701.log.gz 2023-10-18T19:23:28.346 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102380.log 2023-10-18T19:23:28.346 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93047.log.gz 2023-10-18T19:23:28.346 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116213.log 2023-10-18T19:23:28.347 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100015.log.gz 2023-10-18T19:23:28.347 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134242.log 2023-10-18T19:23:28.347 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102380.log.gz 2023-10-18T19:23:28.348 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.116213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91657.log 2023-10-18T19:23:28.348 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116213.log.gz 2023-10-18T19:23:28.348 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104774.log 2023-10-18T19:23:28.348 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134242.log.gz 2023-10-18T19:23:28.349 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95846.log 2023-10-18T19:23:28.349 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91657.log.gz 2023-10-18T19:23:28.349 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92722.log 0.0% 2023-10-18T19:23:28.349 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.104774.log.gz 2023-10-18T19:23:28.350 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95864.log 2023-10-18T19:23:28.350 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95846.log.gz 2023-10-18T19:23:28.350 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118989.log 2023-10-18T19:23:28.351 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92722.log.gz 2023-10-18T19:23:28.351 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111540.log 2023-10-18T19:23:28.351 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95864.log.gz 2023-10-18T19:23:28.351 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106096.log 2023-10-18T19:23:28.352 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118989.log.gz 2023-10-18T19:23:28.352 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121530.log 2023-10-18T19:23:28.352 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111540.log.gz 2023-10-18T19:23:28.352 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106096.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106096.log.gz 2023-10-18T19:23:28.353 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90316.log 2023-10-18T19:23:28.353 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116192.log 2023-10-18T19:23:28.353 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121530.log.gz 2023-10-18T19:23:28.353 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.90316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106971.log 2023-10-18T19:23:28.354 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90316.log.gz 2023-10-18T19:23:28.354 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.116192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93756.log 2023-10-18T19:23:28.354 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116192.log.gz 2023-10-18T19:23:28.355 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107504.log 2023-10-18T19:23:28.355 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106971.log.gz 2023-10-18T19:23:28.355 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95602.log 2023-10-18T19:23:28.355 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.93756.log.gz 2023-10-18T19:23:28.356 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131603.log 2023-10-18T19:23:28.356 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107504.log.gz 2023-10-18T19:23:28.356 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101177.log 2023-10-18T19:23:28.356 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95602.log.gz 2023-10-18T19:23:28.357 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130452.log 2023-10-18T19:23:28.357 INFO:teuthology.orchestra.run.smithi082.stderr: 56.5% -- replaced with /var/log/ceph/ceph-client.admin.131603.log.gz 2023-10-18T19:23:28.357 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97807.log 2023-10-18T19:23:28.357 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101177.log.gz 2023-10-18T19:23:28.358 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133349.log 2023-10-18T19:23:28.358 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130452.log.gz 2023-10-18T19:23:28.358 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114700.log 2023-10-18T19:23:28.358 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97807.log.gz 2023-10-18T19:23:28.359 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96037.log 2023-10-18T19:23:28.359 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133349.log.gz 2023-10-18T19:23:28.359 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121047.log 2023-10-18T19:23:28.360 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114700.log.gz 2023-10-18T19:23:28.360 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125035.log 2023-10-18T19:23:28.360 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96037.log.gz 2023-10-18T19:23:28.360 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93628.log 2023-10-18T19:23:28.361 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121047.log.gz 2023-10-18T19:23:28.361 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138250.log 2023-10-18T19:23:28.361 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125035.log.gz 2023-10-18T19:23:28.361 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117958.log 2023-10-18T19:23:28.362 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93628.log.gz 2023-10-18T19:23:28.362 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.138250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92702.log 2023-10-18T19:23:28.362 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.138250.log.gz 2023-10-18T19:23:28.363 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.117958.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-18T19:23:28.363 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117958.log.gz 2023-10-18T19:23:28.363 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113690.log 2023-10-18T19:23:28.363 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92702.log.gz 2023-10-18T19:23:28.364 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139449.log 2023-10-18T19:23:28.364 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118658.log 2023-10-18T19:23:28.364 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113690.log.gz 2023-10-18T19:23:28.364 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.139449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129320.log 2023-10-18T19:23:28.365 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.139449.log.gz 2023-10-18T19:23:28.365 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91635.log 2023-10-18T19:23:28.365 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118658.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118658.log.gz 2023-10-18T19:23:28.365 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133653.log 2023-10-18T19:23:28.366 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129320.log.gz 2023-10-18T19:23:28.366 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91635.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91635.log.gz 2023-10-18T19:23:28.366 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140968.log 2023-10-18T19:23:28.367 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 /var/log/ceph/ceph-client.admin.133653.log: --verbose -- /var/log/ceph/ceph-client.admin.113523.log 2023-10-18T19:23:28.367 INFO:teuthology.orchestra.run.smithi082.stderr: 10.2% -- replaced with /var/log/ceph/ceph-client.admin.133653.log.gz 2023-10-18T19:23:28.367 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.140968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115130.log 2023-10-18T19:23:28.367 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.140968.log.gz 2023-10-18T19:23:28.368 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133898.log 0.0% 2023-10-18T19:23:28.368 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.113523.log.gz 2023-10-18T19:23:28.368 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99575.log 2023-10-18T19:23:28.368 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115130.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115130.log.gz 2023-10-18T19:23:28.369 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110785.log 2023-10-18T19:23:28.369 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133898.log.gz 2023-10-18T19:23:28.369 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99575.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99575.log.gz 2023-10-18T19:23:28.369 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127058.log 2023-10-18T19:23:28.370 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134156.log 2023-10-18T19:23:28.370 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110785.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110785.log.gz 2023-10-18T19:23:28.370 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109270.log 2023-10-18T19:23:28.371 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127058.log.gz 2023-10-18T19:23:28.371 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134156.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134156.log.gzgzip 2023-10-18T19:23:28.371 INFO:teuthology.orchestra.run.smithi082.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.130688.log 2023-10-18T19:23:28.371 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121853.log 2023-10-18T19:23:28.372 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109270.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109270.log.gz 2023-10-18T19:23:28.372 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123726.log 2023-10-18T19:23:28.372 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130688.log.gz 2023-10-18T19:23:28.372 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121853.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.121853.log.gz 2023-10-18T19:23:28.373 INFO:teuthology.orchestra.run.smithi082.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.120127.log 2023-10-18T19:23:28.373 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118575.log 2023-10-18T19:23:28.373 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123726.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123726.log.gz 2023-10-18T19:23:28.373 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141698.log 2023-10-18T19:23:28.374 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120127.log.gz 2023-10-18T19:23:28.374 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118575.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118575.log.gz 2023-10-18T19:23:28.374 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100718.log 2023-10-18T19:23:28.374 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118843.log 2023-10-18T19:23:28.375 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.141698.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.141698.log.gz 2023-10-18T19:23:28.375 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102569.log 2023-10-18T19:23:28.375 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100718.log.gz 2023-10-18T19:23:28.375 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118843.log: gzip -5 --verbose -- 0.0%/var/log/ceph/ceph-client.admin.116063.log 2023-10-18T19:23:28.376 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.118843.log.gz 2023-10-18T19:23:28.376 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101126.log 2023-10-18T19:23:28.376 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102569.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102569.log.gz 2023-10-18T19:23:28.377 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.116063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111104.log 2023-10-18T19:23:28.377 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116063.log.gz 2023-10-18T19:23:28.377 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101126.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101126.log.gz 2023-10-18T19:23:28.377 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130958.log 2023-10-18T19:23:28.378 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101721.log 2023-10-18T19:23:28.378 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111104.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111104.log.gz 2023-10-18T19:23:28.378 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97484.log 2023-10-18T19:23:28.379 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130958.log.gz 2023-10-18T19:23:28.379 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101721.log: gzip 0.0%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.101721.log.gz-- 2023-10-18T19:23:28.379 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131434.log 2023-10-18T19:23:28.379 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142042.log 2023-10-18T19:23:28.381 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97484.log.gz 2023-10-18T19:23:28.381 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124321.log 2023-10-18T19:23:28.382 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131434.log.gz 2023-10-18T19:23:28.382 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.142042.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.142042.log.gz 2023-10-18T19:23:28.382 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135478.log 2023-10-18T19:23:28.382 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108408.log 2023-10-18T19:23:28.383 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124321.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124321.log.gz 2023-10-18T19:23:28.383 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112821.log 2023-10-18T19:23:28.383 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135478.log.gz 2023-10-18T19:23:28.383 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108408.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108408.log.gz 2023-10-18T19:23:28.384 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110291.log 2023-10-18T19:23:28.384 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112755.log 2023-10-18T19:23:28.384 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112821.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112821.log.gz 2023-10-18T19:23:28.384 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96099.log 2023-10-18T19:23:28.385 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110291.log.gz 2023-10-18T19:23:28.385 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112755.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112755.log.gz 2023-10-18T19:23:28.385 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124401.log 2023-10-18T19:23:28.385 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120214.log 2023-10-18T19:23:28.386 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96099.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96099.log.gz 2023-10-18T19:23:28.386 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119955.log 2023-10-18T19:23:28.386 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124401.log.gz 2023-10-18T19:23:28.387 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92740.log 2023-10-18T19:23:28.387 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120214.log.gz 2023-10-18T19:23:28.387 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98028.log 2023-10-18T19:23:28.387 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119955.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119955.log.gz 2023-10-18T19:23:28.388 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99735.log 2023-10-18T19:23:28.388 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92740.log.gz 2023-10-18T19:23:28.388 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94893.log 2023-10-18T19:23:28.388 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.98028.log.gz 2023-10-18T19:23:28.389 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119783.log 2023-10-18T19:23:28.389 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99735.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99735.log.gz 2023-10-18T19:23:28.389 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128908.log 2023-10-18T19:23:28.389 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94893.log.gz 2023-10-18T19:23:28.390 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122918.log 2023-10-18T19:23:28.390 INFO:teuthology.orchestra.run.smithi082.stderr: 22.1% -- replaced with /var/log/ceph/ceph-client.admin.119783.log.gz 2023-10-18T19:23:28.390 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107711.log 2023-10-18T19:23:28.391 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128908.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128908.log.gz 2023-10-18T19:23:28.391 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114851.log 2023-10-18T19:23:28.391 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122918.log.gz 2023-10-18T19:23:28.391 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117571.log 2023-10-18T19:23:28.392 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107711.log.gz 2023-10-18T19:23:28.392 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98045.log 2023-10-18T19:23:28.392 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114851.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114851.log.gz 2023-10-18T19:23:28.392 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.117571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110506.log 2023-10-18T19:23:28.393 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117571.log.gz 2023-10-18T19:23:28.393 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113606.log 2023-10-18T19:23:28.393 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.98045.log.gz 2023-10-18T19:23:28.393 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118435.log 2023-10-18T19:23:28.394 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110506.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110506.log.gz 2023-10-18T19:23:28.394 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120170.log 2023-10-18T19:23:28.394 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113606.log.gz 2023-10-18T19:23:28.394 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118435.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118435.log.gzgzip 2023-10-18T19:23:28.395 INFO:teuthology.orchestra.run.smithi082.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.104580.log 2023-10-18T19:23:28.395 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104623.log 2023-10-18T19:23:28.395 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120170.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120170.log.gz 2023-10-18T19:23:28.396 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100905.log 2023-10-18T19:23:28.396 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104580.log.gz 2023-10-18T19:23:28.396 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92514.log 0.0% 2023-10-18T19:23:28.396 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.104623.log.gz 2023-10-18T19:23:28.397 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140539.log 2023-10-18T19:23:28.397 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100905.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100905.log.gz 2023-10-18T19:23:28.397 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99255.log 2023-10-18T19:23:28.397 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92514.log.gz 2023-10-18T19:23:28.398 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.140539.log: gzip -5 --verbose 0.0%-- -- replaced with /var/log/ceph/ceph-client.admin.140539.log.gz/var/log/ceph/ceph-client.admin.122323.log 2023-10-18T19:23:28.398 INFO:teuthology.orchestra.run.smithi082.stderr: 2023-10-18T19:23:28.398 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92245.log 2023-10-18T19:23:28.399 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99255.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99255.log.gz 2023-10-18T19:23:28.399 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95409.log 2023-10-18T19:23:28.399 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122323.log.gz 2023-10-18T19:23:28.399 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92245.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92245.log.gz 2023-10-18T19:23:28.399 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104325.log 2023-10-18T19:23:28.400 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113711.log 2023-10-18T19:23:28.400 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95409.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95409.log.gz 2023-10-18T19:23:28.400 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105722.log 2023-10-18T19:23:28.401 INFO:teuthology.orchestra.run.smithi082.stderr: 22.5% -- replaced with /var/log/ceph/ceph-client.admin.104325.log.gz 2023-10-18T19:23:28.401 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106853.log 2023-10-18T19:23:28.401 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113711.log.gz 2023-10-18T19:23:28.401 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134940.log 2023-10-18T19:23:28.401 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105722.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105722.log.gz 2023-10-18T19:23:28.402 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123198.log 2023-10-18T19:23:28.402 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106853.log.gz 2023-10-18T19:23:28.402 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134940.log: gzip 0.0%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.134940.log.gz 2023-10-18T19:23:28.403 INFO:teuthology.orchestra.run.smithi082.stderr:-- /var/log/ceph/ceph-client.admin.121573.log 2023-10-18T19:23:28.403 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115568.log 2023-10-18T19:23:28.403 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123198.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123198.log.gz 2023-10-18T19:23:28.403 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121573.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121573.log.gz 2023-10-18T19:23:28.404 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124583.log 2023-10-18T19:23:28.404 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115568.log: gzip -5 --verbose 0.0%-- /var/log/ceph/ceph-client.admin.94077.log -- replaced with /var/log/ceph/ceph-client.admin.115568.log.gz 2023-10-18T19:23:28.404 INFO:teuthology.orchestra.run.smithi082.stderr: 2023-10-18T19:23:28.404 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124583.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124583.log.gz 2023-10-18T19:23:28.404 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114119.log 2023-10-18T19:23:28.405 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90400.log 2023-10-18T19:23:28.405 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94077.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94077.log.gz 2023-10-18T19:23:28.405 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122789.log 2023-10-18T19:23:28.406 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114119.log.gz 2023-10-18T19:23:28.406 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.90400.log: gzip -5 --verbose 0.0%-- -- replaced with /var/log/ceph/ceph-client.admin.90400.log.gz/var/log/ceph/ceph-client.admin.109599.log 2023-10-18T19:23:28.406 INFO:teuthology.orchestra.run.smithi082.stderr: 2023-10-18T19:23:28.406 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116170.log 2023-10-18T19:23:28.407 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122789.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122789.log.gz 2023-10-18T19:23:28.407 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127641.log 2023-10-18T19:23:28.407 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109599.log.gz 2023-10-18T19:23:28.407 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.116170.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.116170.log.gz-5 2023-10-18T19:23:28.408 INFO:teuthology.orchestra.run.smithi082.stderr:--verbose -- /var/log/ceph/ceph-client.admin.129167.log 2023-10-18T19:23:28.408 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98487.log 2023-10-18T19:23:28.408 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127641.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127641.log.gz 2023-10-18T19:23:28.409 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100786.log 2023-10-18T19:23:28.409 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129167.log.gz 2023-10-18T19:23:28.409 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130731.log 2023-10-18T19:23:28.409 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.98487.log.gz 2023-10-18T19:23:28.410 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100786.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100786.log.gz 2023-10-18T19:23:28.410 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100837.log 2023-10-18T19:23:28.410 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126513.log 2023-10-18T19:23:28.410 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130731.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130731.log.gz 2023-10-18T19:23:28.411 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99755.log 2023-10-18T19:23:28.411 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100837.log.gz 2023-10-18T19:23:28.411 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126513.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126513.log.gz 2023-10-18T19:23:28.411 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127823.log 2023-10-18T19:23:28.412 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107814.log 2023-10-18T19:23:28.412 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99755.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99755.log.gz 2023-10-18T19:23:28.412 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106771.log 2023-10-18T19:23:28.412 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127823.log.gz 2023-10-18T19:23:28.413 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107814.log: gzip 0.0%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.107814.log.gz 2023-10-18T19:23:28.413 INFO:teuthology.orchestra.run.smithi082.stderr:-- /var/log/ceph/ceph-client.admin.103968.log 2023-10-18T19:23:28.413 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108042.log 2023-10-18T19:23:28.414 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106771.log: 57.1% -- replaced with /var/log/ceph/ceph-client.admin.106771.log.gz 2023-10-18T19:23:28.414 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110871.log 2023-10-18T19:23:28.414 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103968.log.gz 2023-10-18T19:23:28.414 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108042.log: gzip -5 --verbose 0.0%-- -- replaced with /var/log/ceph/ceph-client.admin.108042.log.gz/var/log/ceph/ceph-client.admin.133043.log 2023-10-18T19:23:28.415 INFO:teuthology.orchestra.run.smithi082.stderr: 2023-10-18T19:23:28.415 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99375.log 2023-10-18T19:23:28.415 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110871.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110871.log.gz 2023-10-18T19:23:28.415 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98266.log 2023-10-18T19:23:28.416 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133043.log.gz 2023-10-18T19:23:28.416 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99375.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99375.log.gzgzip 2023-10-18T19:23:28.416 INFO:teuthology.orchestra.run.smithi082.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.109107.log 2023-10-18T19:23:28.416 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110828.log 2023-10-18T19:23:28.417 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98266.log: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.98266.log.gz 2023-10-18T19:23:28.417 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118044.log 2023-10-18T19:23:28.417 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109107.log.gz 2023-10-18T19:23:28.418 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110828.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110828.log.gz 2023-10-18T19:23:28.418 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127228.log 2023-10-18T19:23:28.418 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118044.log: gzip 0.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.118044.log.gz--verbose 2023-10-18T19:23:28.418 INFO:teuthology.orchestra.run.smithi082.stderr:-- /var/log/ceph/ceph-client.admin.113196.log 2023-10-18T19:23:28.419 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127228.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127228.log.gz 2023-10-18T19:23:28.419 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98655.log 2023-10-18T19:23:28.419 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113483.log 2023-10-18T19:23:28.419 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113196.log.gz 2023-10-18T19:23:28.420 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108808.log 2023-10-18T19:23:28.420 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98655.log.gz 2023-10-18T19:23:28.420 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122132.log 2023-10-18T19:23:28.420 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113483.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113483.log.gz 2023-10-18T19:23:28.421 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139323.log 2023-10-18T19:23:28.421 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108808.log.gz 2023-10-18T19:23:28.421 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122132.log: gzip -5 --verbose -- 0.0%/var/log/ceph/ceph-client.admin.110115.log 2023-10-18T19:23:28.422 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.122132.log.gz 2023-10-18T19:23:28.422 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131560.log 2023-10-18T19:23:28.422 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.139323.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.139323.log.gz 2023-10-18T19:23:28.422 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142252.log 2023-10-18T19:23:28.423 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110115.log.gz 2023-10-18T19:23:28.423 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131560.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.131560.log.gz 2023-10-18T19:23:28.423 INFO:teuthology.orchestra.run.smithi082.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.121381.log 2023-10-18T19:23:28.423 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112299.log 2023-10-18T19:23:28.424 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.142252.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.142252.log.gz 2023-10-18T19:23:28.424 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91872.log 2023-10-18T19:23:28.424 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121381.log.gz 2023-10-18T19:23:28.424 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112299.log: gzip 0.0%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.112299.log.gz-- 2023-10-18T19:23:28.425 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100235.log 2023-10-18T19:23:28.425 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98419.log 2023-10-18T19:23:28.425 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91872.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91872.log.gz 2023-10-18T19:23:28.426 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125572.log 2023-10-18T19:23:28.426 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100235.log.gz 2023-10-18T19:23:28.426 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128888.log 2023-10-18T19:23:28.426 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.98419.log.gz 2023-10-18T19:23:28.427 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114765.log 2023-10-18T19:23:28.427 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125572.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125572.log.gz 2023-10-18T19:23:28.427 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101041.log 2023-10-18T19:23:28.427 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128888.log.gz 2023-10-18T19:23:28.428 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114765.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114765.log.gz 2023-10-18T19:23:28.428 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129875.log 2023-10-18T19:23:28.428 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105182.log 2023-10-18T19:23:28.428 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101041.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101041.log.gz 2023-10-18T19:23:28.429 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97467.log 2023-10-18T19:23:28.429 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129875.log.gz 2023-10-18T19:23:28.429 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111718.log 2023-10-18T19:23:28.430 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105182.log.gz 2023-10-18T19:23:28.430 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94785.log 2023-10-18T19:23:28.430 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97467.log: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97467.log.gz 2023-10-18T19:23:28.430 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92400.log 2023-10-18T19:23:28.431 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111718.log.gz 2023-10-18T19:23:28.431 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94785.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94785.log.gz 2023-10-18T19:23:28.431 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94335.log 2023-10-18T19:23:28.431 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128726.log 2023-10-18T19:23:28.432 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92400.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92400.log.gz 2023-10-18T19:23:28.432 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92760.log 2023-10-18T19:23:28.432 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94335.log.gz 2023-10-18T19:23:28.432 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128726.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128726.log.gz 2023-10-18T19:23:28.433 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98317.log 2023-10-18T19:23:28.433 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102027.log 2023-10-18T19:23:28.433 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92760.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92760.log.gz 2023-10-18T19:23:28.433 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105616.log 2023-10-18T19:23:28.434 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.98317.log.gz 2023-10-18T19:23:28.434 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113009.log 2023-10-18T19:23:28.434 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102027.log.gz 2023-10-18T19:23:28.435 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102918.log 2023-10-18T19:23:28.435 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105616.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105616.log.gz 2023-10-18T19:23:28.435 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125766.log 2023-10-18T19:23:28.435 INFO:teuthology.orchestra.run.smithi082.stderr: 22.4% -- replaced with /var/log/ceph/ceph-client.admin.113009.log.gz 2023-10-18T19:23:28.436 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134919.log 2023-10-18T19:23:28.436 INFO:teuthology.orchestra.run.smithi082.stderr: 22.8% -- replaced with /var/log/ceph/ceph-client.admin.102918.log.gz 2023-10-18T19:23:28.436 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96077.log 2023-10-18T19:23:28.436 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125766.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125766.log.gz 2023-10-18T19:23:28.437 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98283.log 2023-10-18T19:23:28.437 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134919.log.gz 2023-10-18T19:23:28.437 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96077.log: gzip -5 --verbose 0.0%-- -- replaced with /var/log/ceph/ceph-client.admin.96077.log.gz/var/log/ceph/ceph-client.admin.125293.log 2023-10-18T19:23:28.438 INFO:teuthology.orchestra.run.smithi082.stderr: 2023-10-18T19:23:28.438 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130010.log 2023-10-18T19:23:28.438 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98283.log: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.98283.log.gz 2023-10-18T19:23:28.438 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120623.log 2023-10-18T19:23:28.439 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125293.log.gz 2023-10-18T19:23:28.439 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130010.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130010.log.gz 2023-10-18T19:23:28.439 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114248.log 2023-10-18T19:23:28.439 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103548.log 2023-10-18T19:23:28.440 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120623.log.gz 2023-10-18T19:23:28.440 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132119.log 2023-10-18T19:23:28.440 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114248.log.gz 2023-10-18T19:23:28.441 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100495.log 2023-10-18T19:23:28.441 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103548.log.gz 2023-10-18T19:23:28.441 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125099.log 2023-10-18T19:23:28.441 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132119.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132119.log.gz 2023-10-18T19:23:28.442 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125357.log 2023-10-18T19:23:28.442 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100495.log.gz 2023-10-18T19:23:28.442 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125099.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125099.log.gz 2023-10-18T19:23:28.442 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134006.log 2023-10-18T19:23:28.443 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113320.log 2023-10-18T19:23:28.443 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125357.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125357.log.gz 2023-10-18T19:23:28.443 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119891.log 2023-10-18T19:23:28.443 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134006.log.gz 2023-10-18T19:23:28.444 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123550.log 0.0% 2023-10-18T19:23:28.444 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.113320.log.gz 2023-10-18T19:23:28.444 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123515.log 2023-10-18T19:23:28.445 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119891.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119891.log.gz 2023-10-18T19:23:28.445 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107212.log 2023-10-18T19:23:28.445 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123550.log.gz 2023-10-18T19:23:28.445 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123515.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123515.log.gzgzip 2023-10-18T19:23:28.446 INFO:teuthology.orchestra.run.smithi082.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.101381.log 2023-10-18T19:23:28.446 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92880.log 2023-10-18T19:23:28.446 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107212.log: 57.1% -- replaced with /var/log/ceph/ceph-client.admin.107212.log.gz 2023-10-18T19:23:28.446 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133287.log 2023-10-18T19:23:28.447 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101381.log.gz 2023-10-18T19:23:28.447 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92880.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92880.log.gz 2023-10-18T19:23:28.447 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94957.log 2023-10-18T19:23:28.448 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91179.log 2023-10-18T19:23:28.448 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133287.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133287.log.gz 2023-10-18T19:23:28.448 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120644.log 2023-10-18T19:23:28.448 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94957.log.gz 2023-10-18T19:23:28.449 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91179.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91179.log.gz 2023-10-18T19:23:28.449 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107294.log 2023-10-18T19:23:28.449 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135689.log 2023-10-18T19:23:28.449 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120644.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120644.log.gz 2023-10-18T19:23:28.450 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141438.log 2023-10-18T19:23:28.450 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107294.log.gz 2023-10-18T19:23:28.450 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120838.log 2023-10-18T19:23:28.450 INFO:teuthology.orchestra.run.smithi082.stderr: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.135689.log.gz 2023-10-18T19:23:28.451 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130088.log 2023-10-18T19:23:28.451 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.141438.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.141438.log.gz 2023-10-18T19:23:28.451 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98915.log 2023-10-18T19:23:28.451 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120838.log.gz 2023-10-18T19:23:28.452 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130088.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130088.log.gz 2023-10-18T19:23:28.452 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116277.log 2023-10-18T19:23:28.452 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121424.log 2023-10-18T19:23:28.453 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98915.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98915.log.gz 2023-10-18T19:23:28.453 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.116277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99835.log 2023-10-18T19:23:28.453 INFO:teuthology.orchestra.run.smithi082.stderr: 3.8% -- replaced with /var/log/ceph/ceph-client.admin.116277.log.gz 2023-10-18T19:23:28.453 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108647.log 2023-10-18T19:23:28.454 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121424.log.gz 2023-10-18T19:23:28.454 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101891.log 2023-10-18T19:23:28.454 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99835.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99835.log.gz 2023-10-18T19:23:28.454 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127541.log 0.0% 2023-10-18T19:23:28.455 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.108647.log.gz 2023-10-18T19:23:28.455 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101891.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101891.log.gz 2023-10-18T19:23:28.455 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93851.log 2023-10-18T19:23:28.455 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128250.log 2023-10-18T19:23:28.456 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127541.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127541.log.gz 2023-10-18T19:23:28.456 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111890.log 2023-10-18T19:23:28.456 INFO:teuthology.orchestra.run.smithi082.stderr: 5.3% -- replaced with /var/log/ceph/ceph-client.admin.93851.log.gz 2023-10-18T19:23:28.456 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123849.log 2023-10-18T19:23:28.457 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128250.log.gz 2023-10-18T19:23:28.457 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126767.log 2023-10-18T19:23:28.457 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111890.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111890.log.gz 2023-10-18T19:23:28.458 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107009.log 2023-10-18T19:23:28.458 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123849.log.gz 2023-10-18T19:23:28.458 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126767.log: gzip 0.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.126767.log.gz--verbose 2023-10-18T19:23:28.458 INFO:teuthology.orchestra.run.smithi082.stderr:-- /var/log/ceph/ceph-client.admin.107444.log 2023-10-18T19:23:28.459 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103149.log 2023-10-18T19:23:28.459 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107009.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107009.log.gz 2023-10-18T19:23:28.459 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133120.log 2023-10-18T19:23:28.459 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107444.log.gz 2023-10-18T19:23:28.460 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103149.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103149.log.gz 2023-10-18T19:23:28.460 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101245.log 2023-10-18T19:23:28.460 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119977.log 2023-10-18T19:23:28.461 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133120.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133120.log.gz 2023-10-18T19:23:28.461 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109707.log 2023-10-18T19:23:28.461 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101245.log.gz 2023-10-18T19:23:28.461 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97127.log 2023-10-18T19:23:28.462 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119977.log.gz 2023-10-18T19:23:28.462 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126304.log 2023-10-18T19:23:28.462 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109707.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109707.log.gz 2023-10-18T19:23:28.462 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97093.log 2023-10-18T19:23:28.463 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97127.log.gz 2023-10-18T19:23:28.463 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112127.log 2023-10-18T19:23:28.463 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126304.log.gz 2023-10-18T19:23:28.464 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96357.log 2023-10-18T19:23:28.464 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97093.log: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97093.log.gz 2023-10-18T19:23:28.464 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110613.log 2023-10-18T19:23:28.464 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112127.log.gz 2023-10-18T19:23:28.465 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96357.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96357.log.gz 2023-10-18T19:23:28.465 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112041.log 2023-10-18T19:23:28.465 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120365.log 2023-10-18T19:23:28.465 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110613.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110613.log.gz 2023-10-18T19:23:28.466 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104157.log 2023-10-18T19:23:28.466 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112041.log.gz 2023-10-18T19:23:28.466 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120365.log: 0.0%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.120365.log.gz--verbose 2023-10-18T19:23:28.466 INFO:teuthology.orchestra.run.smithi082.stderr:-- /var/log/ceph/ceph-client.admin.111000.log 2023-10-18T19:23:28.467 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126188.log 2023-10-18T19:23:28.467 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104157.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104157.log.gz 2023-10-18T19:23:28.467 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105596.log 2023-10-18T19:23:28.468 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111000.log.gz 2023-10-18T19:23:28.468 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129654.log 2023-10-18T19:23:28.468 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126188.log.gz 2023-10-18T19:23:28.468 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133984.log 2023-10-18T19:23:28.469 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105596.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105596.log.gz 2023-10-18T19:23:28.469 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100769.log 2023-10-18T19:23:28.469 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129654.log.gz 2023-10-18T19:23:28.469 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133984.log: gzip -5 --verbose 0.0%-- /var/log/ceph/ceph-client.admin.137743.log -- replaced with /var/log/ceph/ceph-client.admin.133984.log.gz 2023-10-18T19:23:28.470 INFO:teuthology.orchestra.run.smithi082.stderr: 2023-10-18T19:23:28.470 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108148.log 2023-10-18T19:23:28.470 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100769.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100769.log.gz 2023-10-18T19:23:28.471 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.137743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108105.log 2023-10-18T19:23:28.471 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.137743.log.gz 2023-10-18T19:23:28.471 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121089.log 2023-10-18T19:23:28.471 INFO:teuthology.orchestra.run.smithi082.stderr: 22.8% -- replaced with /var/log/ceph/ceph-client.admin.108148.log.gz 2023-10-18T19:23:28.471 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90484.log 2023-10-18T19:23:28.472 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108105.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108105.log.gz 2023-10-18T19:23:28.472 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100667.log 2023-10-18T19:23:28.472 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121089.log.gz 2023-10-18T19:23:28.472 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.90484.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90484.log.gz 2023-10-18T19:23:28.473 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97875.log 2023-10-18T19:23:28.473 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142996.log 2023-10-18T19:23:28.473 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100667.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100667.log.gz 2023-10-18T19:23:28.474 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123219.log 2023-10-18T19:23:28.474 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97875.log.gz 2023-10-18T19:23:28.474 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.142996.log: gzip -5 --verbose -- 0.0%/var/log/ceph/ceph-client.admin.110592.log -- replaced with /var/log/ceph/ceph-client.admin.142996.log.gz 2023-10-18T19:23:28.474 INFO:teuthology.orchestra.run.smithi082.stderr: 2023-10-18T19:23:28.475 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129224.log 2023-10-18T19:23:28.475 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123219.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123219.log.gz 2023-10-18T19:23:28.475 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125981.log 2023-10-18T19:23:28.475 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110592.log.gz 2023-10-18T19:23:28.476 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129224.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129224.log.gz 2023-10-18T19:23:28.476 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113840.log 2023-10-18T19:23:28.476 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126152.log 2023-10-18T19:23:28.477 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125981.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125981.log.gz 2023-10-18T19:23:28.477 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111847.log 2023-10-18T19:23:28.477 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113840.log.gz 2023-10-18T19:23:28.478 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126152.log: gzip 0.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.126152.log.gz--verbose 2023-10-18T19:23:28.478 INFO:teuthology.orchestra.run.smithi082.stderr:-- /var/log/ceph/ceph-client.admin.106933.log 2023-10-18T19:23:28.478 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91678.log 2023-10-18T19:23:28.478 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111847.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111847.log.gz 2023-10-18T19:23:28.479 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128210.log 2023-10-18T19:23:28.479 INFO:teuthology.orchestra.run.smithi082.stderr: 22.4% -- replaced with /var/log/ceph/ceph-client.admin.106933.log.gz 2023-10-18T19:23:28.479 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91678.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91678.log.gz 2023-10-18T19:23:28.480 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114097.log 2023-10-18T19:23:28.480 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98011.log 2023-10-18T19:23:28.480 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128210.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128210.log.gz 2023-10-18T19:23:28.480 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109943.log 2023-10-18T19:23:28.481 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114097.log.gz 2023-10-18T19:23:28.481 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114571.log 2023-10-18T19:23:28.481 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.98011.log.gz 2023-10-18T19:23:28.482 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115762.log 2023-10-18T19:23:28.482 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109943.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109943.log.gz 2023-10-18T19:23:28.482 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109664.log 2023-10-18T19:23:28.482 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114571.log.gz 2023-10-18T19:23:28.483 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115762.log: gzip 0.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.115762.log.gz--verbose 2023-10-18T19:23:28.483 INFO:teuthology.orchestra.run.smithi082.stderr:-- /var/log/ceph/ceph-client.admin.92362.log 2023-10-18T19:23:28.483 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123788.log 2023-10-18T19:23:28.484 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109664.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109664.log.gz 2023-10-18T19:23:28.484 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139029.log 2023-10-18T19:23:28.484 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92362.log.gz 2023-10-18T19:23:28.484 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90190.log 2023-10-18T19:23:28.485 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123788.log.gz 2023-10-18T19:23:28.485 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113503.log 2023-10-18T19:23:28.485 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.139029.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.139029.log.gz 2023-10-18T19:23:28.485 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.90190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108788.log 2023-10-18T19:23:28.486 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90190.log.gz 2023-10-18T19:23:28.486 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113503.log: gzip -5 0.0%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.113503.log.gz 2023-10-18T19:23:28.486 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133533.log 2023-10-18T19:23:28.486 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95258.log 2023-10-18T19:23:28.487 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108788.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108788.log.gz 2023-10-18T19:23:28.487 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96249.log 2023-10-18T19:23:28.487 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133533.log.gz 2023-10-18T19:23:28.487 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95258.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95258.log.gz 2023-10-18T19:23:28.488 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101483.log 2023-10-18T19:23:28.488 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128434.log 2023-10-18T19:23:28.488 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96249.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96249.log.gz 2023-10-18T19:23:28.488 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133613.log 2023-10-18T19:23:28.489 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101483.log.gz 2023-10-18T19:23:28.489 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132592.log 2023-10-18T19:23:28.489 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128434.log.gz 2023-10-18T19:23:28.490 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127880.log 2023-10-18T19:23:28.490 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133613.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133613.log.gz 2023-10-18T19:23:28.490 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93219.log 2023-10-18T19:23:28.490 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132592.log.gz 2023-10-18T19:23:28.491 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127880.log: 0.0%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.127880.log.gz--verbose 2023-10-18T19:23:28.491 INFO:teuthology.orchestra.run.smithi082.stderr:-- /var/log/ceph/ceph-client.admin.141480.log 2023-10-18T19:23:28.491 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125078.log 2023-10-18T19:23:28.491 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93219.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93219.log.gz 2023-10-18T19:23:28.492 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.141480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109922.log 2023-10-18T19:23:28.492 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.141480.log.gz 2023-10-18T19:23:28.492 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125078.log: gzip 0.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.125078.log.gz--verbose 2023-10-18T19:23:28.492 INFO:teuthology.orchestra.run.smithi082.stderr:-- /var/log/ceph/ceph-client.admin.134500.log 2023-10-18T19:23:28.493 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109922.log: /var/log/ceph/ceph-client.admin.101211.log 2023-10-18T19:23:28.493 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109922.log.gz 2023-10-18T19:23:28.493 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134500.log: gzip 0.0%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.134500.log.gz-- 2023-10-18T19:23:28.493 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122089.log 2023-10-18T19:23:28.494 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101211.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101211.log.gz 2023-10-18T19:23:28.494 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119590.log 2023-10-18T19:23:28.494 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92816.log 2023-10-18T19:23:28.495 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122089.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122089.log.gz 2023-10-18T19:23:28.495 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108351.log 2023-10-18T19:23:28.495 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119590.log.gz 2023-10-18T19:23:28.495 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92816.log: gzip -5 --verbose -- 0.0%/var/log/ceph/ceph-client.admin.96486.log -- replaced with /var/log/ceph/ceph-client.admin.92816.log.gz 2023-10-18T19:23:28.496 INFO:teuthology.orchestra.run.smithi082.stderr: 2023-10-18T19:23:28.496 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130344.log 2023-10-18T19:23:28.496 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108351.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108351.log.gz 2023-10-18T19:23:28.496 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131105.log 2023-10-18T19:23:28.497 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96486.log.gz 2023-10-18T19:23:28.497 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104262.log 2023-10-18T19:23:28.497 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130344.log.gz 2023-10-18T19:23:28.497 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130194.log 2023-10-18T19:23:28.498 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131105.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131105.log.gz 2023-10-18T19:23:28.498 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124605.log 2023-10-18T19:23:28.498 INFO:teuthology.orchestra.run.smithi082.stderr: 23.6% -- replaced with /var/log/ceph/ceph-client.admin.104262.log.gz 2023-10-18T19:23:28.499 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102792.log 2023-10-18T19:23:28.499 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130194.log.gz 2023-10-18T19:23:28.499 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134983.log 2023-10-18T19:23:28.499 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124605.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124605.log.gz 2023-10-18T19:23:28.500 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133247.log 2023-10-18T19:23:28.500 INFO:teuthology.orchestra.run.smithi082.stderr: 22.8% -- replaced with /var/log/ceph/ceph-client.admin.102792.log.gz 2023-10-18T19:23:28.500 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115934.log 2023-10-18T19:23:28.500 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134983.log.gz 2023-10-18T19:23:28.501 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110244.log 2023-10-18T19:23:28.501 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133247.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133247.log.gz 2023-10-18T19:23:28.501 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96464.log 2023-10-18T19:23:28.501 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115934.log.gz 2023-10-18T19:23:28.502 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110244.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.110244.log.gz-5 2023-10-18T19:23:28.502 INFO:teuthology.orchestra.run.smithi082.stderr:--verbose -- /var/log/ceph/ceph-client.admin.100115.log 2023-10-18T19:23:28.502 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 /var/log/ceph/ceph-client.admin.96464.log: --verbose -- /var/log/ceph/ceph-client.admin.113990.log 2023-10-18T19:23:28.502 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96464.log.gz 2023-10-18T19:23:28.503 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100115.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100115.log.gz 2023-10-18T19:23:28.503 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127207.log 2023-10-18T19:23:28.503 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113990.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113990.log.gz 2023-10-18T19:23:28.504 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120687.log 2023-10-18T19:23:28.504 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128860.log 2023-10-18T19:23:28.504 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127207.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127207.log.gz 2023-10-18T19:23:28.504 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95623.log 2023-10-18T19:23:28.505 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120687.log.gz 2023-10-18T19:23:28.505 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128860.log: gzip 0.0%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.128860.log.gz 2023-10-18T19:23:28.505 INFO:teuthology.orchestra.run.smithi082.stderr:-- /var/log/ceph/ceph-client.admin.97552.log 2023-10-18T19:23:28.505 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92684.log 2023-10-18T19:23:28.506 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95623.log: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.95623.log.gz 2023-10-18T19:23:28.506 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132571.log 2023-10-18T19:23:28.506 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97552.log.gz 2023-10-18T19:23:28.507 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92684.log: gzip 0.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.92684.log.gz--verbose 2023-10-18T19:23:28.507 INFO:teuthology.orchestra.run.smithi082.stderr:-- /var/log/ceph/ceph-client.admin.108606.log 2023-10-18T19:23:28.507 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96636.log 2023-10-18T19:23:28.507 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132571.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132571.log.gz 2023-10-18T19:23:28.508 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128609.log 2023-10-18T19:23:28.508 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108606.log.gz 2023-10-18T19:23:28.508 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128650.log 2023-10-18T19:23:28.508 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96636.log.gz 2023-10-18T19:23:28.509 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124239.log 2023-10-18T19:23:28.509 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128609.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128609.log.gz 2023-10-18T19:23:28.509 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125637.log 2023-10-18T19:23:28.510 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128650.log.gz 2023-10-18T19:23:28.510 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124239.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.124239.log.gz 2023-10-18T19:23:28.510 INFO:teuthology.orchestra.run.smithi082.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.108449.log 2023-10-18T19:23:28.510 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101738.log 2023-10-18T19:23:28.511 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125637.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125637.log.gz 2023-10-18T19:23:28.511 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135069.log 2023-10-18T19:23:28.511 INFO:teuthology.orchestra.run.smithi082.stderr: 22.9% -- replaced with /var/log/ceph/ceph-client.admin.108449.log.gz 2023-10-18T19:23:28.511 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134628.log 2023-10-18T19:23:28.512 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101738.log.gz 2023-10-18T19:23:28.512 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139281.log 2023-10-18T19:23:28.512 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135069.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135069.log.gz 2023-10-18T19:23:28.512 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124042.log 2023-10-18T19:23:28.513 INFO:teuthology.orchestra.run.smithi082.stderr: 22.4% -- replaced with /var/log/ceph/ceph-client.admin.134628.log.gz 2023-10-18T19:23:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.139281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134457.log 2023-10-18T19:23:28.513 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.139281.log.gz 2023-10-18T19:23:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118969.log 2023-10-18T19:23:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124042.log: 11.6% -- replaced with /var/log/ceph/ceph-client.admin.124042.log.gz 2023-10-18T19:23:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121257.log 2023-10-18T19:23:28.514 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134457.log.gz 2023-10-18T19:23:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142391.log 2023-10-18T19:23:28.515 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118969.log.gz 2023-10-18T19:23:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109900.log 2023-10-18T19:23:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121257.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121257.log.gz 2023-10-18T19:23:28.516 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.142391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103107.log 2023-10-18T19:23:28.516 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.142391.log.gz 2023-10-18T19:23:28.516 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109900.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109900.log.gz 2023-10-18T19:23:28.516 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91248.log 2023-10-18T19:23:28.517 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108746.log 2023-10-18T19:23:28.517 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103107.log: 21.7% -- replaced with /var/log/ceph/ceph-client.admin.103107.log.gz 2023-10-18T19:23:28.517 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101347.log 2023-10-18T19:23:28.517 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91248.log.gz 2023-10-18T19:23:28.518 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122478.log 2023-10-18T19:23:28.518 INFO:teuthology.orchestra.run.smithi082.stderr: 23.2% -- replaced with /var/log/ceph/ceph-client.admin.108746.log.gz 2023-10-18T19:23:28.518 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124507.log 2023-10-18T19:23:28.518 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101347.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101347.log.gz 2023-10-18T19:23:28.519 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120859.log 2023-10-18T19:23:28.519 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122478.log.gz 2023-10-18T19:23:28.519 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112967.log 2023-10-18T19:23:28.520 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124507.log.gz 2023-10-18T19:23:28.520 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94249.log 2023-10-18T19:23:28.520 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120859.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120859.log.gz 2023-10-18T19:23:28.520 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102299.log 2023-10-18T19:23:28.521 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112967.log.gz 2023-10-18T19:23:28.521 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94249.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.94249.log.gz-5 2023-10-18T19:23:28.521 INFO:teuthology.orchestra.run.smithi082.stderr:--verbose -- /var/log/ceph/ceph-client.admin.119440.log 2023-10-18T19:23:28.521 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119259.log 2023-10-18T19:23:28.522 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102299.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102299.log.gz 2023-10-18T19:23:28.522 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99335.log 2023-10-18T19:23:28.522 INFO:teuthology.orchestra.run.smithi082.stderr: 22.8% -- replaced with /var/log/ceph/ceph-client.admin.119440.log.gz 2023-10-18T19:23:28.523 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110008.log 2023-10-18T19:23:28.523 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.119259.log.gz 2023-10-18T19:23:28.523 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112256.log 2023-10-18T19:23:28.523 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99335.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99335.log.gz 2023-10-18T19:23:28.524 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123963.log 2023-10-18T19:23:28.524 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110008.log.gz 2023-10-18T19:23:28.524 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112256.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112256.log.gz 2023-10-18T19:23:28.524 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133877.log 2023-10-18T19:23:28.525 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128352.log 2023-10-18T19:23:28.525 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123963.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123963.log.gz 2023-10-18T19:23:28.525 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133877.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.133877.log.gz-5 2023-10-18T19:23:28.525 INFO:teuthology.orchestra.run.smithi082.stderr:--verbose -- /var/log/ceph/ceph-client.admin.93790.log 2023-10-18T19:23:28.526 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128352.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128352.log.gz 2023-10-18T19:23:28.526 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95811.log 2023-10-18T19:23:28.526 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125895.log 2023-10-18T19:23:28.527 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93790.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93790.log.gz 2023-10-18T19:23:28.527 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94120.log 2023-10-18T19:23:28.527 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95811.log.gz 2023-10-18T19:23:28.527 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125895.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125895.log.gz 2023-10-18T19:23:28.528 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-18T19:23:28.528 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124444.log 2023-10-18T19:23:28.528 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94120.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94120.log.gz 2023-10-18T19:23:28.528 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101653.log 2023-10-18T19:23:28.529 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107960.log 2023-10-18T19:23:28.529 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124444.log.gz 2023-10-18T19:23:28.529 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134522.log 2023-10-18T19:23:28.529 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101653.log.gz 2023-10-18T19:23:28.530 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108929.log 2023-10-18T19:23:28.530 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107960.log.gz 2023-10-18T19:23:28.530 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97042.log 2023-10-18T19:23:28.531 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134522.log.gz 2023-10-18T19:23:28.531 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133771.log 2023-10-18T19:23:28.531 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108929.log.gz 2023-10-18T19:23:28.531 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103443.log 2023-10-18T19:23:28.532 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97042.log.gz 2023-10-18T19:23:28.532 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107753.log 2023-10-18T19:23:28.532 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133771.log.gz 2023-10-18T19:23:28.532 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96786.log 2023-10-18T19:23:28.533 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103443.log.gz 2023-10-18T19:23:28.533 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134436.log 2023-10-18T19:23:28.533 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107753.log.gz 2023-10-18T19:23:28.533 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100939.log 2023-10-18T19:23:28.534 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96786.log.gz 2023-10-18T19:23:28.534 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130172.log 2023-10-18T19:23:28.534 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134436.log.gz 2023-10-18T19:23:28.535 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101840.log 2023-10-18T19:23:28.535 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100939.log.gz 2023-10-18T19:23:28.535 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94378.log 2023-10-18T19:23:28.535 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130172.log.gz 2023-10-18T19:23:28.536 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94270.log 2023-10-18T19:23:28.536 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101840.log.gz 2023-10-18T19:23:28.536 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100871.log 2023-10-18T19:23:28.536 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94378.log.gz 2023-10-18T19:23:28.537 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94013.log 2023-10-18T19:23:28.537 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94270.log.gz 2023-10-18T19:23:28.537 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98300.log 2023-10-18T19:23:28.537 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100871.log.gz 2023-10-18T19:23:28.538 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122832.log 2023-10-18T19:23:28.538 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94013.log.gz 2023-10-18T19:23:28.538 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121069.log 2023-10-18T19:23:28.539 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.98300.log.gz 2023-10-18T19:23:28.539 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142349.log 2023-10-18T19:23:28.539 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122832.log.gz 2023-10-18T19:23:28.539 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124487.log 2023-10-18T19:23:28.540 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121069.log.gz 2023-10-18T19:23:28.540 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.142349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95215.log 2023-10-18T19:23:28.540 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.142349.log.gz 2023-10-18T19:23:28.540 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113135.log 2023-10-18T19:23:28.541 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124487.log.gz 2023-10-18T19:23:28.541 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114679.log 2023-10-18T19:23:28.541 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95215.log.gz 2023-10-18T19:23:28.541 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128783.log 2023-10-18T19:23:28.542 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113135.log.gz 2023-10-18T19:23:28.542 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126399.log 2023-10-18T19:23:28.542 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114679.log.gz 2023-10-18T19:23:28.543 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141958.log 2023-10-18T19:23:28.543 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128783.log.gz 2023-10-18T19:23:28.543 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94764.log 2023-10-18T19:23:28.543 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126399.log.gz 2023-10-18T19:23:28.544 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.141958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116903.log 2023-10-18T19:23:28.544 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.141958.log.gz 2023-10-18T19:23:28.544 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132807.log 2023-10-18T19:23:28.544 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94764.log.gz 2023-10-18T19:23:28.545 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.116903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97671.log 2023-10-18T19:23:28.545 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116903.log.gz 2023-10-18T19:23:28.545 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108371.log 2023-10-18T19:23:28.545 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132807.log.gz 2023-10-18T19:23:28.546 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109287.log 2023-10-18T19:23:28.546 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97671.log.gz 2023-10-18T19:23:28.546 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106246.log 2023-10-18T19:23:28.547 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.108371.log.gz 2023-10-18T19:23:28.547 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140581.log 2023-10-18T19:23:28.547 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109287.log.gz 2023-10-18T19:23:28.547 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119336.log 2023-10-18T19:23:28.548 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106246.log.gz 2023-10-18T19:23:28.548 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.140581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114958.log 2023-10-18T19:23:28.548 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.140581.log.gz 2023-10-18T19:23:28.548 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135616.log 2023-10-18T19:23:28.549 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119336.log.gz 2023-10-18T19:23:28.549 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125744.log 2023-10-18T19:23:28.549 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114958.log.gz 2023-10-18T19:23:28.550 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105323.log 2023-10-18T19:23:28.550 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135616.log.gz 2023-10-18T19:23:28.550 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105457.log 2023-10-18T19:23:28.550 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125744.log.gz 2023-10-18T19:23:28.551 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122068.log 2023-10-18T19:23:28.551 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105323.log.gz 2023-10-18T19:23:28.551 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100435.log 2023-10-18T19:23:28.551 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105457.log.gz 2023-10-18T19:23:28.552 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119611.log 2023-10-18T19:23:28.552 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122068.log.gz 2023-10-18T19:23:28.552 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110635.log 2023-10-18T19:23:28.552 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100435.log.gz 2023-10-18T19:23:28.553 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114226.log 2023-10-18T19:23:28.553 INFO:teuthology.orchestra.run.smithi082.stderr: 23.0% -- replaced with /var/log/ceph/ceph-client.admin.119611.log.gz 2023-10-18T19:23:28.553 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130366.log 2023-10-18T19:23:28.553 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110635.log.gz 2023-10-18T19:23:28.554 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99315.log 2023-10-18T19:23:28.554 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114226.log.gz 2023-10-18T19:23:28.554 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131668.log 2023-10-18T19:23:28.555 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130366.log.gz 2023-10-18T19:23:28.555 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113216.log 2023-10-18T19:23:28.555 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99315.log.gz 2023-10-18T19:23:28.555 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132227.log 2023-10-18T19:23:28.556 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131668.log.gz 2023-10-18T19:23:28.556 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112363.log 2023-10-18T19:23:28.556 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113216.log.gz 2023-10-18T19:23:28.556 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113968.log 2023-10-18T19:23:28.557 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132227.log.gz 2023-10-18T19:23:28.557 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138502.log 2023-10-18T19:23:28.557 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112363.log.gz 2023-10-18T19:23:28.557 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97059.log 2023-10-18T19:23:28.558 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113968.log.gz 2023-10-18T19:23:28.558 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.138502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105534.log 2023-10-18T19:23:28.558 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.138502.log.gz 2023-10-18T19:23:28.559 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140187.log 2023-10-18T19:23:28.559 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97059.log.gz 2023-10-18T19:23:28.559 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97722.log 2023-10-18T19:23:28.559 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105534.log.gz 2023-10-18T19:23:28.560 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.140187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119654.log 2023-10-18T19:23:28.560 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.140187.log.gz 2023-10-18T19:23:28.560 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128570.log 2023-10-18T19:23:28.560 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97722.log.gz 2023-10-18T19:23:28.561 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104838.log 2023-10-18T19:23:28.561 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119654.log.gz 2023-10-18T19:23:28.561 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89925.log 2023-10-18T19:23:28.561 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128570.log.gz 2023-10-18T19:23:28.562 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94592.log 2023-10-18T19:23:28.562 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104838.log.gz 2023-10-18T19:23:28.562 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.89925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110072.log 2023-10-18T19:23:28.563 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89925.log.gz 2023-10-18T19:23:28.563 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134704.log 2023-10-18T19:23:28.563 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94592.log.gz 2023-10-18T19:23:28.563 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114312.log 2023-10-18T19:23:28.564 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110072.log.gz 2023-10-18T19:23:28.564 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96889.log 2023-10-18T19:23:28.564 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134704.log.gz 2023-10-18T19:23:28.564 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117721.log 2023-10-18T19:23:28.565 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114312.log.gz 2023-10-18T19:23:28.565 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133573.log 2023-10-18T19:23:28.565 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.96889.log.gz 2023-10-18T19:23:28.565 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.117721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104494.log 2023-10-18T19:23:28.566 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117721.log.gz 2023-10-18T19:23:28.566 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101568.log 2023-10-18T19:23:28.566 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133573.log.gz 2023-10-18T19:23:28.566 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112342.log 2023-10-18T19:23:28.567 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104494.log.gz 2023-10-18T19:23:28.567 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118555.log 2023-10-18T19:23:28.567 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101568.log.gz 2023-10-18T19:23:28.567 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105851.log 2023-10-18T19:23:28.568 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112342.log.gz 2023-10-18T19:23:28.568 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125207.log 2023-10-18T19:23:28.568 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118555.log.gz 2023-10-18T19:23:28.569 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135241.log 2023-10-18T19:23:28.569 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105851.log.gz 2023-10-18T19:23:28.569 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93563.log 2023-10-18T19:23:28.569 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125207.log.gz 2023-10-18T19:23:28.570 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92182.log 2023-10-18T19:23:28.570 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135241.log.gz 2023-10-18T19:23:28.570 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119462.log 2023-10-18T19:23:28.571 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93563.log.gz 2023-10-18T19:23:28.571 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95452.log 2023-10-18T19:23:28.571 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92182.log.gz 2023-10-18T19:23:28.571 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102485.log 2023-10-18T19:23:28.572 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119462.log.gz 2023-10-18T19:23:28.572 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91700.log 2023-10-18T19:23:28.572 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95452.log.gz 2023-10-18T19:23:28.572 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125142.log 2023-10-18T19:23:28.573 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102485.log.gz 2023-10-18T19:23:28.573 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101619.log 2023-10-18T19:23:28.573 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91700.log.gz 2023-10-18T19:23:28.573 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124755.log 2023-10-18T19:23:28.574 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125142.log.gz 2023-10-18T19:23:28.574 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97110.log 2023-10-18T19:23:28.574 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101619.log.gz 2023-10-18T19:23:28.575 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131475.log 2023-10-18T19:23:28.575 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124755.log.gz 2023-10-18T19:23:28.575 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118679.log 2023-10-18T19:23:28.575 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97110.log.gz 2023-10-18T19:23:28.576 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115190.log 2023-10-18T19:23:28.576 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131475.log.gz 2023-10-18T19:23:28.576 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111245.log 2023-10-18T19:23:28.576 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118679.log.gz 2023-10-18T19:23:28.577 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111367.log 2023-10-18T19:23:28.577 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115190.log.gz 2023-10-18T19:23:28.577 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141052.log 2023-10-18T19:23:28.578 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111245.log.gz 2023-10-18T19:23:28.578 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98555.log 2023-10-18T19:23:28.578 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111367.log.gz 2023-10-18T19:23:28.578 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.141052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137898.log 2023-10-18T19:23:28.579 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.141052.log.gz 2023-10-18T19:23:28.579 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124820.log 2023-10-18T19:23:28.579 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.98555.log.gz 2023-10-18T19:23:28.579 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.137898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93606.log 2023-10-18T19:23:28.580 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.137898.log.gz 2023-10-18T19:23:28.580 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115442.log 2023-10-18T19:23:28.580 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124820.log.gz 2023-10-18T19:23:28.580 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100650.log 2023-10-18T19:23:28.581 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93606.log.gz 2023-10-18T19:23:28.581 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100195.log 2023-10-18T19:23:28.581 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115442.log.gz 2023-10-18T19:23:28.582 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129089.log 2023-10-18T19:23:28.582 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100650.log.gz 2023-10-18T19:23:28.582 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129674.log 2023-10-18T19:23:28.582 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100195.log.gz 2023-10-18T19:23:28.583 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105304.log 2023-10-18T19:23:28.583 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129089.log.gz 2023-10-18T19:23:28.583 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106375.log 2023-10-18T19:23:28.585 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129674.log.gz 2023-10-18T19:23:28.585 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126625.log 2023-10-18T19:23:28.585 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105304.log.gz 2023-10-18T19:23:28.586 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127682.log 2023-10-18T19:23:28.586 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106375.log.gz 2023-10-18T19:23:28.586 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124906.log 2023-10-18T19:23:28.587 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126625.log.gz 2023-10-18T19:23:28.587 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109128.log 2023-10-18T19:23:28.587 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127682.log.gz 2023-10-18T19:23:28.587 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121213.log 2023-10-18T19:23:28.588 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124906.log.gz 2023-10-18T19:23:28.588 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105361.log 2023-10-18T19:23:28.588 INFO:teuthology.orchestra.run.smithi082.stderr: 22.4% -- replaced with /var/log/ceph/ceph-client.admin.109128.log.gz 2023-10-18T19:23:28.588 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124138.log 2023-10-18T19:23:28.589 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121213.log.gz 2023-10-18T19:23:28.589 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126688.log 2023-10-18T19:23:28.589 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105361.log.gz 2023-10-18T19:23:28.589 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100135.log 2023-10-18T19:23:28.590 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124138.log.gz 2023-10-18T19:23:28.590 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90442.log 2023-10-18T19:23:28.590 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126688.log.gz 2023-10-18T19:23:28.591 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97943.log 2023-10-18T19:23:28.591 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100135.log.gz 2023-10-18T19:23:28.591 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.90442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105636.log 2023-10-18T19:23:28.591 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90442.log.gz 2023-10-18T19:23:28.592 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129417.log 2023-10-18T19:23:28.592 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97943.log.gz 2023-10-18T19:23:28.592 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113091.log 2023-10-18T19:23:28.592 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105636.log.gz 2023-10-18T19:23:28.593 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134748.log 2023-10-18T19:23:28.593 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129417.log.gz 2023-10-18T19:23:28.593 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115740.log 2023-10-18T19:23:28.593 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113091.log.gz 2023-10-18T19:23:28.594 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137800.log 2023-10-18T19:23:28.594 INFO:teuthology.orchestra.run.smithi082.stderr: 56.1% -- replaced with /var/log/ceph/ceph-client.admin.134748.log.gz 2023-10-18T19:23:28.594 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110850.log 2023-10-18T19:23:28.595 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115740.log.gz 2023-10-18T19:23:28.595 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.137800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106954.log 2023-10-18T19:23:28.595 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.137800.log.gz 2023-10-18T19:23:28.595 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99055.log 2023-10-18T19:23:28.596 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110850.log.gz 2023-10-18T19:23:28.596 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108689.log 2023-10-18T19:23:28.596 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106954.log.gz 2023-10-18T19:23:28.596 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135599.log 2023-10-18T19:23:28.597 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99055.log.gz 2023-10-18T19:23:28.597 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130710.log 2023-10-18T19:23:28.597 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108689.log.gz 2023-10-18T19:23:28.597 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103359.log 2023-10-18T19:23:28.598 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135599.log.gz 2023-10-18T19:23:28.598 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118375.log 2023-10-18T19:23:28.598 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130710.log.gz 2023-10-18T19:23:28.599 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140707.log 2023-10-18T19:23:28.599 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103359.log.gz 2023-10-18T19:23:28.599 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121466.log 2023-10-18T19:23:28.599 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118375.log.gz 2023-10-18T19:23:28.600 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.140707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125787.log 2023-10-18T19:23:28.600 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.140707.log.gz 2023-10-18T19:23:28.600 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117614.log 2023-10-18T19:23:28.600 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121466.log.gz 2023-10-18T19:23:28.601 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134285.log 2023-10-18T19:23:28.601 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125787.log.gz 2023-10-18T19:23:28.601 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.117614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107370.log 2023-10-18T19:23:28.602 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117614.log.gz 2023-10-18T19:23:28.602 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121616.log 2023-10-18T19:23:28.602 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134285.log.gz 2023-10-18T19:23:28.602 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118130.log 2023-10-18T19:23:28.603 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107370.log.gz 2023-10-18T19:23:28.603 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112213.log 2023-10-18T19:23:28.603 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121616.log.gz 2023-10-18T19:23:28.603 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93542.log 2023-10-18T19:23:28.604 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118130.log.gz 2023-10-18T19:23:28.604 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118455.log 2023-10-18T19:23:28.604 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112213.log.gz 2023-10-18T19:23:28.604 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127745.log 2023-10-18T19:23:28.605 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93542.log.gz 2023-10-18T19:23:28.605 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127561.log 2023-10-18T19:23:28.605 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118455.log.gz 2023-10-18T19:23:28.605 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106813.log 2023-10-18T19:23:28.606 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127745.log.gz 2023-10-18T19:23:28.606 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118863.log 2023-10-18T19:23:28.606 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127561.log.gz 2023-10-18T19:23:28.607 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127427.log 2023-10-18T19:23:28.607 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106813.log.gz 2023-10-18T19:23:28.607 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113585.log 2023-10-18T19:23:28.607 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118863.log.gz 2023-10-18T19:23:28.608 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121767.log 2023-10-18T19:23:28.608 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127427.log.gz 2023-10-18T19:23:28.608 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91291.log 2023-10-18T19:23:28.608 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113585.log.gz 2023-10-18T19:23:28.609 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103947.log 2023-10-18T19:23:28.609 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121767.log.gz 2023-10-18T19:23:28.609 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122678.log 2023-10-18T19:23:28.609 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91291.log.gz 2023-10-18T19:23:28.610 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113341.log 2023-10-18T19:23:28.610 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103947.log.gz 2023-10-18T19:23:28.610 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127136.log 2023-10-18T19:23:28.611 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122678.log.gz 2023-10-18T19:23:28.611 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133474.log 2023-10-18T19:23:28.611 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113341.log.gz 2023-10-18T19:23:28.611 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100990.log 2023-10-18T19:23:28.612 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127136.log.gz 2023-10-18T19:23:28.612 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98695.log 2023-10-18T19:23:28.612 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133474.log.gz 2023-10-18T19:23:28.612 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101755.log 2023-10-18T19:23:28.613 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100990.log.gz 2023-10-18T19:23:28.613 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94163.log 2023-10-18T19:23:28.613 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98695.log.gz 2023-10-18T19:23:28.613 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96443.log 2023-10-18T19:23:28.614 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101755.log.gz 2023-10-18T19:23:28.614 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103065.log 2023-10-18T19:23:28.614 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94163.log.gz 2023-10-18T19:23:28.615 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96443.log: gzip -5 --verbose -- 0.0%/var/log/ceph/ceph-client.admin.132248.log 2023-10-18T19:23:28.615 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.96443.log.gz 2023-10-18T19:23:28.615 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103065.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103065.log.gz 2023-10-18T19:23:28.615 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137575.log 2023-10-18T19:23:28.616 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111998.log 2023-10-18T19:23:28.616 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132248.log.gz 2023-10-18T19:23:28.616 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.137575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109428.log 2023-10-18T19:23:28.616 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.137575.log.gz 2023-10-18T19:23:28.617 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133203.log 2023-10-18T19:23:28.617 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111998.log.gz 2023-10-18T19:23:28.617 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127444.log 2023-10-18T19:23:28.617 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109428.log.gz 2023-10-18T19:23:28.618 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129574.log 2023-10-18T19:23:28.618 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133203.log.gz 2023-10-18T19:23:28.618 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123629.log 2023-10-18T19:23:28.619 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127444.log.gz 2023-10-18T19:23:28.619 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96378.log 2023-10-18T19:23:28.619 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129574.log.gz 2023-10-18T19:23:28.619 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133963.log 2023-10-18T19:23:28.620 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123629.log.gz 2023-10-18T19:23:28.620 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92590.log 2023-10-18T19:23:28.620 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96378.log.gz 2023-10-18T19:23:28.620 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127330.log 2023-10-18T19:23:28.621 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133963.log.gz 2023-10-18T19:23:28.621 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139071.log 2023-10-18T19:23:28.621 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92590.log.gz 2023-10-18T19:23:28.622 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111192.log 2023-10-18T19:23:28.622 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127330.log.gz 2023-10-18T19:23:28.622 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.139071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102464.log 2023-10-18T19:23:28.622 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.139071.log.gz 2023-10-18T19:23:28.623 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135134.log 2023-10-18T19:23:28.623 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111192.log.gz 2023-10-18T19:23:28.623 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113048.log 2023-10-18T19:23:28.623 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102464.log.gz 2023-10-18T19:23:28.624 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131328.log 2023-10-18T19:23:28.624 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135134.log.gz 2023-10-18T19:23:28.624 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140876.log 2023-10-18T19:23:28.625 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113048.log.gz 2023-10-18T19:23:28.625 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113776.log 2023-10-18T19:23:28.625 INFO:teuthology.orchestra.run.smithi082.stderr: 56.1% -- replaced with /var/log/ceph/ceph-client.admin.131328.log.gz 2023-10-18T19:23:28.625 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.140876.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.140876.log.gz 2023-10-18T19:23:28.626 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113776.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113776.log.gz 2023-10-18T19:23:28.626 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101058.log 2023-10-18T19:23:28.626 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102611.log 2023-10-18T19:23:28.626 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111826.log 2023-10-18T19:23:28.627 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101058.log.gz 2023-10-18T19:23:28.627 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130047.log 2023-10-18T19:23:28.627 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102611.log.gz 2023-10-18T19:23:28.628 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94051.log 2023-10-18T19:23:28.628 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111826.log.gz 2023-10-18T19:23:28.628 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131369.log 2023-10-18T19:23:28.628 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130047.log.gz 2023-10-18T19:23:28.629 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102960.log 2023-10-18T19:23:28.629 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94051.log.gz 2023-10-18T19:23:28.629 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86922.log 2023-10-18T19:23:28.629 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131369.log.gz 2023-10-18T19:23:28.630 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91334.log 2023-10-18T19:23:28.630 INFO:teuthology.orchestra.run.smithi082.stderr: 22.8% -- replaced with /var/log/ceph/ceph-client.admin.102960.log.gz 2023-10-18T19:23:28.630 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.86922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98755.log 2023-10-18T19:23:28.631 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86922.log.gz 2023-10-18T19:23:28.631 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142484.log 2023-10-18T19:23:28.631 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91334.log.gz 2023-10-18T19:23:28.631 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129130.log 2023-10-18T19:23:28.632 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98755.log.gz 2023-10-18T19:23:28.632 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.142484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102197.log 2023-10-18T19:23:28.632 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.142484.log.gz 2023-10-18T19:23:28.633 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99075.log 2023-10-18T19:23:28.633 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129130.log.gz 2023-10-18T19:23:28.633 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139197.log 2023-10-18T19:23:28.633 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102197.log.gz 2023-10-18T19:23:28.634 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104881.log 2023-10-18T19:23:28.634 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99075.log.gz 2023-10-18T19:23:28.634 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.139197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104537.log 2023-10-18T19:23:28.634 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.139197.log.gz 2023-10-18T19:23:28.635 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118259.log 2023-10-18T19:23:28.635 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104881.log.gz 2023-10-18T19:23:28.635 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128093.log 2023-10-18T19:23:28.636 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104537.log.gz 2023-10-18T19:23:28.636 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98675.log 2023-10-18T19:23:28.636 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118259.log.gz 2023-10-18T19:23:28.636 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130301.log 2023-10-18T19:23:28.637 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128093.log.gz 2023-10-18T19:23:28.637 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103296.log 2023-10-18T19:23:28.637 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98675.log.gz 2023-10-18T19:23:28.637 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98635.log 2023-10-18T19:23:28.638 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130301.log.gz 2023-10-18T19:23:28.638 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120580.log 2023-10-18T19:23:28.638 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103296.log.gz 2023-10-18T19:23:28.638 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98521.log 2023-10-18T19:23:28.639 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98635.log.gz 2023-10-18T19:23:28.639 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97008.log 2023-10-18T19:23:28.639 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120580.log.gz 2023-10-18T19:23:28.640 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115869.log 2023-10-18T19:23:28.640 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.98521.log.gz 2023-10-18T19:23:28.640 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123942.log 2023-10-18T19:23:28.640 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97008.log.gz 2023-10-18T19:23:28.641 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123608.log 2023-10-18T19:23:28.641 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115869.log.gz 2023-10-18T19:23:28.641 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126243.log 2023-10-18T19:23:28.641 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123942.log.gz 2023-10-18T19:23:28.642 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98385.log 2023-10-18T19:23:28.642 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123608.log.gz 2023-10-18T19:23:28.642 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114937.log 2023-10-18T19:23:28.642 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126243.log.gz 2023-10-18T19:23:28.643 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116947.log 2023-10-18T19:23:28.643 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.98385.log.gz 2023-10-18T19:23:28.643 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119869.log 2023-10-18T19:23:28.644 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114937.log.gz 2023-10-18T19:23:28.644 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.116947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99855.log 2023-10-18T19:23:28.644 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116947.log.gz 2023-10-18T19:23:28.644 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128803.log 2023-10-18T19:23:28.645 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119869.log.gz 2023-10-18T19:23:28.645 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118216.log 2023-10-18T19:23:28.645 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99855.log.gz 2023-10-18T19:23:28.645 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97926.log 2023-10-18T19:23:28.646 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128803.log.gz 2023-10-18T19:23:28.646 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104666.log 2023-10-18T19:23:28.646 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118216.log.gz 2023-10-18T19:23:28.647 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102590.log 2023-10-18T19:23:28.647 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97926.log.gz 2023-10-18T19:23:28.647 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118759.log 2023-10-18T19:23:28.647 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104666.log.gz 2023-10-18T19:23:28.648 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95967.log 2023-10-18T19:23:28.648 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102590.log.gz 2023-10-18T19:23:28.648 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110441.log 2023-10-18T19:23:28.648 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118759.log.gz 2023-10-18T19:23:28.649 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112884.log 2023-10-18T19:23:28.649 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95967.log.gz 2023-10-18T19:23:28.649 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126110.log 2023-10-18T19:23:28.650 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110441.log.gz 2023-10-18T19:23:28.650 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93413.log 2023-10-18T19:23:28.650 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112884.log.gz 2023-10-18T19:23:28.650 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140103.log 2023-10-18T19:23:28.651 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126110.log.gz 2023-10-18T19:23:28.651 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132442.log 2023-10-18T19:23:28.651 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93413.log.gz 2023-10-18T19:23:28.651 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.140103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113925.log 2023-10-18T19:23:28.652 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.140103.log.gz 2023-10-18T19:23:28.652 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100055.log 2023-10-18T19:23:28.652 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132442.log.gz 2023-10-18T19:23:28.653 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134584.log 2023-10-18T19:23:28.653 INFO:teuthology.orchestra.run.smithi082.stderr: 23.7% -- replaced with /var/log/ceph/ceph-client.admin.113925.log.gz 2023-10-18T19:23:28.653 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124381.log 2023-10-18T19:23:28.653 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100055.log.gz 2023-10-18T19:23:28.654 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115023.log 2023-10-18T19:23:28.654 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134584.log.gz 2023-10-18T19:23:28.654 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123827.log 2023-10-18T19:23:28.654 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124381.log.gz 2023-10-18T19:23:28.655 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110957.log 2023-10-18T19:23:28.655 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115023.log.gz 2023-10-18T19:23:28.655 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98538.log 2023-10-18T19:23:28.655 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123827.log.gz 2023-10-18T19:23:28.656 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110312.log 2023-10-18T19:23:28.656 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110957.log.gz 2023-10-18T19:23:28.656 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122360.log 2023-10-18T19:23:28.657 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.98538.log.gz 2023-10-18T19:23:28.657 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107898.log 2023-10-18T19:23:28.657 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110312.log.gz 2023-10-18T19:23:28.657 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130516.log 2023-10-18T19:23:28.658 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122360.log.gz 2023-10-18T19:23:28.658 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121111.log 2023-10-18T19:23:28.658 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107898.log.gz 2023-10-18T19:23:28.658 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101143.log 2023-10-18T19:23:28.659 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130516.log.gz 2023-10-18T19:23:28.659 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131990.log 2023-10-18T19:23:28.659 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121111.log.gz 2023-10-18T19:23:28.660 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114743.log 2023-10-18T19:23:28.660 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101143.log.gz 2023-10-18T19:23:28.660 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106160.log 2023-10-18T19:23:28.660 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131990.log.gz 2023-10-18T19:23:28.661 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113947.log 2023-10-18T19:23:28.661 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114743.log.gz 2023-10-18T19:23:28.661 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101449.log 2023-10-18T19:23:28.661 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106160.log.gz 2023-10-18T19:23:28.662 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127484.log 2023-10-18T19:23:28.662 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113947.log.gz 2023-10-18T19:23:28.662 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107671.log 2023-10-18T19:23:28.662 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101449.log.gz 2023-10-18T19:23:28.663 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121788.log 2023-10-18T19:23:28.663 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127484.log.gz 2023-10-18T19:23:28.663 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114033.log 2023-10-18T19:23:28.663 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107671.log.gz 2023-10-18T19:23:28.664 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124970.log 2023-10-18T19:23:28.664 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121788.log.gz 2023-10-18T19:23:28.664 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105679.log 2023-10-18T19:23:28.664 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114033.log.gz 2023-10-18T19:23:28.665 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107349.log 2023-10-18T19:23:28.665 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124970.log.gz 2023-10-18T19:23:28.665 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133432.log 2023-10-18T19:23:28.666 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105679.log.gz 2023-10-18T19:23:28.666 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126916.log 2023-10-18T19:23:28.666 INFO:teuthology.orchestra.run.smithi082.stderr: 53.1% -- replaced with /var/log/ceph/ceph-client.admin.107349.log.gz 2023-10-18T19:23:28.666 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127387.log 2023-10-18T19:23:28.667 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133432.log.gz 2023-10-18T19:23:28.667 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138893.log 2023-10-18T19:23:28.667 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126916.log.gz 2023-10-18T19:23:28.667 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113362.log 2023-10-18T19:23:28.668 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127387.log.gz 2023-10-18T19:23:28.668 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.138893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109307.log 2023-10-18T19:23:28.668 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.138893.log.gz 2023-10-18T19:23:28.668 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.113362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98453.log 2023-10-18T19:23:28.669 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113362.log.gz 2023-10-18T19:23:28.669 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128453.log 2023-10-18T19:23:28.669 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109307.log.gz 2023-10-18T19:23:28.670 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94807.log 2023-10-18T19:23:28.670 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98453.log.gz 2023-10-18T19:23:28.670 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94421.log 2023-10-18T19:23:28.670 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128453.log.gz 2023-10-18T19:23:28.671 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140497.log 2023-10-18T19:23:28.671 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94807.log.gz 2023-10-18T19:23:28.671 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137533.log 2023-10-18T19:23:28.671 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94421.log.gz 2023-10-18T19:23:28.672 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.140497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92476.log 2023-10-18T19:23:28.672 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.140497.log.gz 2023-10-18T19:23:28.672 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.137533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99515.log 2023-10-18T19:23:28.672 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.137533.log.gz 2023-10-18T19:23:28.673 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109642.log 2023-10-18T19:23:28.673 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92476.log.gz 2023-10-18T19:23:28.673 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141228.log 2023-10-18T19:23:28.674 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99515.log.gz 2023-10-18T19:23:28.674 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126729.log 2023-10-18T19:23:28.674 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109642.log.gz 2023-10-18T19:23:28.674 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.141228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140749.log 2023-10-18T19:23:28.675 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.141228.log.gz 2023-10-18T19:23:28.675 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110764.log 2023-10-18T19:23:28.675 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126729.log.gz 2023-10-18T19:23:28.675 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.140749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107464.log 2023-10-18T19:23:28.676 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.140749.log.gz 2023-10-18T19:23:28.676 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133834.log 2023-10-18T19:23:28.676 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110764.log.gz 2023-10-18T19:23:28.676 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93262.log 2023-10-18T19:23:28.677 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107464.log.gz 2023-10-18T19:23:28.677 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107093.log 2023-10-18T19:23:28.677 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133834.log.gz 2023-10-18T19:23:28.678 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98615.log 2023-10-18T19:23:28.678 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93262.log.gz 2023-10-18T19:23:28.678 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106466.log 2023-10-18T19:23:28.678 INFO:teuthology.orchestra.run.smithi082.stderr: 23.3% -- replaced with /var/log/ceph/ceph-client.admin.107093.log.gz 2023-10-18T19:23:28.679 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93241.log 2023-10-18T19:23:28.679 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98615.log.gz 2023-10-18T19:23:28.679 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90652.log 2023-10-18T19:23:28.679 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106466.log.gz 2023-10-18T19:23:28.680 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128948.log 2023-10-18T19:23:28.680 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93241.log.gz 2023-10-18T19:23:28.680 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.90652.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90652.log.gz 2023-10-18T19:23:28.680 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97399.log 2023-10-18T19:23:28.681 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125400.log 2023-10-18T19:23:28.681 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128948.log.gz 2023-10-18T19:23:28.681 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128190.log 2023-10-18T19:23:28.681 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97399.log.gz 2023-10-18T19:23:28.682 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93198.log 2023-10-18T19:23:28.682 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125400.log.gz 2023-10-18T19:23:28.682 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105576.log 2023-10-18T19:23:28.683 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128190.log.gz 2023-10-18T19:23:28.683 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101007.log 2023-10-18T19:23:28.683 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93198.log.gz 2023-10-18T19:23:28.683 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122046.log 2023-10-18T19:23:28.684 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105576.log.gz 2023-10-18T19:23:28.684 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114980.log 2023-10-18T19:23:28.684 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101007.log.gz 2023-10-18T19:23:28.684 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124257.log 2023-10-18T19:23:28.685 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122046.log.gz 2023-10-18T19:23:28.685 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132313.log 2023-10-18T19:23:28.685 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114980.log.gz 2023-10-18T19:23:28.685 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97195.log 2023-10-18T19:23:28.686 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124257.log.gz 2023-10-18T19:23:28.686 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101806.log 2023-10-18T19:23:28.686 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132313.log.gz 2023-10-18T19:23:28.687 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97535.log 2023-10-18T19:23:28.687 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97195.log.gz 2023-10-18T19:23:28.687 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96507.log 2023-10-18T19:23:28.687 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101806.log.gz 2023-10-18T19:23:28.688 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127270.log 2023-10-18T19:23:28.688 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97535.log.gz 2023-10-18T19:23:28.688 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119848.log 2023-10-18T19:23:28.688 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96507.log.gz 2023-10-18T19:23:28.689 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132893.log 2023-10-18T19:23:28.689 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127270.log.gz 2023-10-18T19:23:28.689 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100095.log 2023-10-18T19:23:28.689 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119848.log.gz 2023-10-18T19:23:28.690 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105397.log 2023-10-18T19:23:28.690 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132893.log.gz 2023-10-18T19:23:28.690 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.100095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132506.log 2023-10-18T19:23:28.690 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100095.log.gz 2023-10-18T19:23:28.691 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119525.log 2023-10-18T19:23:28.691 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105397.log.gz 2023-10-18T19:23:28.691 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107392.log 2023-10-18T19:23:28.692 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132506.log.gz 2023-10-18T19:23:28.692 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.119525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112453.log 2023-10-18T19:23:28.692 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119525.log.gz 2023-10-18T19:23:28.692 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98815.log 2023-10-18T19:23:28.693 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107392.log.gz 2023-10-18T19:23:28.693 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90232.log 2023-10-18T19:23:28.693 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112453.log.gz 2023-10-18T19:23:28.693 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93773.log 2023-10-18T19:23:28.694 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98815.log.gz 2023-10-18T19:23:28.694 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.90232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98062.log 2023-10-18T19:23:28.694 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90232.log.gz 2023-10-18T19:23:28.694 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117893.log 2023-10-18T19:23:28.695 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93773.log.gz 2023-10-18T19:23:28.695 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122276.log 2023-10-18T19:23:28.695 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.98062.log.gz 2023-10-18T19:23:28.695 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.117893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111333.log 2023-10-18T19:23:28.696 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117893.log.gz 2023-10-18T19:23:28.696 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127076.log 2023-10-18T19:23:28.696 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122276.log.gz 2023-10-18T19:23:28.697 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141832.log 2023-10-18T19:23:28.697 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111333.log.gz 2023-10-18T19:23:28.697 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.127076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93176.log 2023-10-18T19:23:28.697 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127076.log.gz 2023-10-18T19:23:28.698 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.141832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108825.log 2023-10-18T19:23:28.698 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.141832.log.gz 2023-10-18T19:23:28.698 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142610.log 2023-10-18T19:23:28.698 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93176.log.gz 2023-10-18T19:23:28.699 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94678.log 2023-10-18T19:23:28.699 INFO:teuthology.orchestra.run.smithi082.stderr: 22.8% -- replaced with /var/log/ceph/ceph-client.admin.108825.log.gz 2023-10-18T19:23:28.699 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.142610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93155.log 2023-10-18T19:23:28.700 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.142610.log.gz 2023-10-18T19:23:28.700 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118065.log 2023-10-18T19:23:28.700 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94678.log.gz 2023-10-18T19:23:28.700 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123112.log 2023-10-18T19:23:28.701 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93155.log.gz 2023-10-18T19:23:28.701 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107939.log 2023-10-18T19:23:28.701 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118065.log.gz 2023-10-18T19:23:28.701 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128843.log 2023-10-18T19:23:28.702 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123112.log.gz 2023-10-18T19:23:28.702 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124949.log 2023-10-18T19:23:28.702 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107939.log.gz 2023-10-18T19:23:28.702 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130151.log 2023-10-18T19:23:28.703 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128843.log.gz 2023-10-18T19:23:28.703 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.124949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95495.log 2023-10-18T19:23:28.703 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124949.log.gz 2023-10-18T19:23:28.704 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129634.log 2023-10-18T19:23:28.704 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130151.log.gz 2023-10-18T19:23:28.704 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109207.log 2023-10-18T19:23:28.704 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95495.log.gz 2023-10-18T19:23:28.705 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92438.log 2023-10-18T19:23:28.705 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129634.log.gz 2023-10-18T19:23:28.705 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114442.log 2023-10-18T19:23:28.705 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109207.log.gz 2023-10-18T19:23:28.706 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103905.log 2023-10-18T19:23:28.706 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92438.log.gz 2023-10-18T19:23:28.706 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106792.log 2023-10-18T19:23:28.706 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114442.log.gz 2023-10-18T19:23:28.707 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92089.log 2023-10-18T19:23:28.707 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103905.log.gz 2023-10-18T19:23:28.707 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105894.log 2023-10-18T19:23:28.708 INFO:teuthology.orchestra.run.smithi082.stderr: 22.8% -- replaced with /var/log/ceph/ceph-client.admin.106792.log.gz 2023-10-18T19:23:28.708 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129302.log 2023-10-18T19:23:28.708 INFO:teuthology.orchestra.run.smithi082.stderr: 2.5% -- replaced with /var/log/ceph/ceph-client.admin.92089.log.gz 2023-10-18T19:23:28.708 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102010.log 2023-10-18T19:23:28.709 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105894.log.gz 2023-10-18T19:23:28.709 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109449.log 2023-10-18T19:23:28.709 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129302.log.gz 2023-10-18T19:23:28.709 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134790.log 2023-10-18T19:23:28.710 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102010.log.gz 2023-10-18T19:23:28.710 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99535.log 2023-10-18T19:23:28.710 INFO:teuthology.orchestra.run.smithi082.stderr: 23.3% -- replaced with /var/log/ceph/ceph-client.admin.109449.log.gz 2023-10-18T19:23:28.710 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91571.log 2023-10-18T19:23:28.711 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134790.log.gz 2023-10-18T19:23:28.711 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92778.log 2023-10-18T19:23:28.711 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99535.log.gz 2023-10-18T19:23:28.711 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128413.log 2023-10-18T19:23:28.712 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91571.log.gz 2023-10-18T19:23:28.712 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104516.log 2023-10-18T19:23:28.712 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92778.log.gz 2023-10-18T19:23:28.713 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109492.log 2023-10-18T19:23:28.713 INFO:teuthology.orchestra.run.smithi082.stderr: 56.7% -- replaced with /var/log/ceph/ceph-client.admin.128413.log.gz 2023-10-18T19:23:28.713 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121404.log 2023-10-18T19:23:28.713 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104516.log.gz 2023-10-18T19:23:28.714 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120084.log 2023-10-18T19:23:28.714 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109492.log.gz 2023-10-18T19:23:28.714 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141916.log 2023-10-18T19:23:28.714 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121404.log.gz 2023-10-18T19:23:28.715 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.120084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114614.log 2023-10-18T19:23:28.715 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120084.log.gz 2023-10-18T19:23:28.715 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.141916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108251.log 2023-10-18T19:23:28.716 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.141916.log.gz 2023-10-18T19:23:28.716 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98181.log 2023-10-18T19:23:28.716 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114614.log.gz 2023-10-18T19:23:28.716 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105418.log 2023-10-18T19:23:28.717 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.108251.log.gz 2023-10-18T19:23:28.717 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133693.log 2023-10-18T19:23:28.717 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.98181.log.gz 2023-10-18T19:23:28.717 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134070.log 2023-10-18T19:23:28.718 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105418.log.gz 2023-10-18T19:23:28.718 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114722.log 2023-10-18T19:23:28.718 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133693.log.gz 2023-10-18T19:23:28.718 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.134070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126024.log 2023-10-18T19:23:28.719 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134070.log.gz 2023-10-18T19:23:28.719 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97229.log 2023-10-18T19:23:28.719 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114722.log.gz 2023-10-18T19:23:28.719 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123650.log 2023-10-18T19:23:28.720 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126024.log.gz 2023-10-18T19:23:28.720 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116020.log 2023-10-18T19:23:28.720 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97229.log.gz 2023-10-18T19:23:28.721 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109007.log 2023-10-18T19:23:28.721 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123650.log.gz 2023-10-18T19:23:28.721 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.116020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102163.log 2023-10-18T19:23:28.721 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116020.log.gz 2023-10-18T19:23:28.722 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102180.log 2023-10-18T19:23:28.722 INFO:teuthology.orchestra.run.smithi082.stderr: 23.2% -- replaced with /var/log/ceph/ceph-client.admin.109007.log.gz 2023-10-18T19:23:28.722 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106268.log 2023-10-18T19:23:28.722 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102163.log.gz 2023-10-18T19:23:28.723 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101687.log 2023-10-18T19:23:28.723 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102180.log.gz 2023-10-18T19:23:28.723 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94227.log 2023-10-18T19:23:28.723 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106268.log.gz 2023-10-18T19:23:28.724 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133711.log 2023-10-18T19:23:28.724 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101687.log.gz 2023-10-18T19:23:28.724 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121917.log 2023-10-18T19:23:28.724 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94227.log.gz 2023-10-18T19:23:28.725 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117528.log 2023-10-18T19:23:28.725 INFO:teuthology.orchestra.run.smithi082.stderr: 51.1% -- replaced with /var/log/ceph/ceph-client.admin.133711.log.gz 2023-10-18T19:23:28.725 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123568.log 2023-10-18T19:23:28.726 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121917.log.gz 2023-10-18T19:23:28.726 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.117528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98935.log 2023-10-18T19:23:28.726 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117528.log.gz 2023-10-18T19:23:28.726 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.123568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121341.log 2023-10-18T19:23:28.727 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123568.log.gz 2023-10-18T19:23:28.727 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139633.log 2023-10-18T19:23:28.727 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98935.log.gz 2023-10-18T19:23:28.727 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.121341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97365.log 2023-10-18T19:23:28.728 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121341.log.gz 2023-10-18T19:23:28.728 INFO:teuthology.orchestra.run.smithi082.stderr:gzip /var/log/ceph/ceph-client.admin.139633.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.130998.log 2023-10-18T19:23:28.728 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.139633.log.gz 2023-10-18T19:23:28.729 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135542.log 2023-10-18T19:23:28.729 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97365.log.gz 2023-10-18T19:23:28.729 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131818.log 2023-10-18T19:23:28.729 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130998.log.gz 2023-10-18T19:23:28.730 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104860.log 2023-10-18T19:23:28.730 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135542.log.gz 2023-10-18T19:23:28.730 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94485.log 2023-10-18T19:23:28.730 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131818.log.gz 2023-10-18T19:23:28.731 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87331.log 2023-10-18T19:23:28.731 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104860.log.gz 2023-10-18T19:23:28.731 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126805.log 2023-10-18T19:23:28.731 INFO:teuthology.orchestra.run.smithi082.stderr: 27.1% -- replaced with /var/log/ceph/ceph-client.admin.94485.log.gz 2023-10-18T19:23:28.732 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.87331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128552.log 2023-10-18T19:23:28.732 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87331.log.gz 2023-10-18T19:23:28.732 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107423.log 2023-10-18T19:23:28.732 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126805.log.gz 2023-10-18T19:23:28.733 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115357.log 2023-10-18T19:23:28.733 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128552.log.gz 2023-10-18T19:23:28.733 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102729.log 2023-10-18T19:23:28.734 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107423.log.gz 2023-10-18T19:23:28.734 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135633.log 2023-10-18T19:23:28.734 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115357.log.gz 2023-10-18T19:23:28.734 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95712.log 2023-10-18T19:23:28.735 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102729.log.gz 2023-10-18T19:23:28.735 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103002.log 2023-10-18T19:23:28.735 INFO:teuthology.orchestra.run.smithi082.stderr: 10.9% -- replaced with /var/log/ceph/ceph-client.admin.135633.log.gz 2023-10-18T19:23:28.735 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99895.log 2023-10-18T19:23:28.736 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95712.log.gz 2023-10-18T19:23:28.736 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112925.log 2023-10-18T19:23:28.736 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103002.log.gz 2023-10-18T19:23:28.736 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107586.log 2023-10-18T19:23:28.737 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99895.log.gz 2023-10-18T19:23:28.737 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142778.log 2023-10-18T19:23:28.737 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112925.log.gz 2023-10-18T19:23:28.737 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94914.log 2023-10-18T19:23:28.738 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107586.log.gz 2023-10-18T19:23:28.738 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.142778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94936.log 2023-10-18T19:23:28.738 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.142778.log.gz 2023-10-18T19:23:28.739 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126002.log 2023-10-18T19:23:28.739 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94914.log.gz 2023-10-18T19:23:28.739 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96679.log 2023-10-18T19:23:28.739 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94936.log.gz 2023-10-18T19:23:28.740 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.126002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128170.log 2023-10-18T19:23:28.740 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126002.log.gz 2023-10-18T19:23:28.740 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.96679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92900.log 2023-10-18T19:23:28.740 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96679.log.gz 2023-10-18T19:23:28.741 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.128170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108268.log 2023-10-18T19:23:28.741 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128170.log.gz 2023-10-18T19:23:28.741 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91786.log 2023-10-18T19:23:28.741 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92900.log.gz 2023-10-18T19:23:28.742 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102112.log 2023-10-18T19:23:28.742 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108268.log.gz 2023-10-18T19:23:28.742 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.91786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101551.log 2023-10-18T19:23:28.742 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91786.log.gz 2023-10-18T19:23:28.743 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110463.log 2023-10-18T19:23:28.743 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102112.log.gz 2023-10-18T19:23:28.743 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.101551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98735.log 2023-10-18T19:23:28.744 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101551.log.gz 2023-10-18T19:23:28.744 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104388.log 2023-10-18T19:23:28.744 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110463.log.gz 2023-10-18T19:23:28.744 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93090.log 2023-10-18T19:23:28.745 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98735.log.gz 2023-10-18T19:23:28.745 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132721.log 2023-10-18T19:23:28.745 INFO:teuthology.orchestra.run.smithi082.stderr: 22.5% -- replaced with /var/log/ceph/ceph-client.admin.104388.log.gz 2023-10-18T19:23:28.745 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.93090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102359.log 2023-10-18T19:23:28.746 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93090.log.gz 2023-10-18T19:23:28.746 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.132721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125551.log 2023-10-18T19:23:28.746 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132721.log.gz 2023-10-18T19:23:28.746 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.102359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105284.log 2023-10-18T19:23:28.747 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102359.log.gz 2023-10-18T19:23:28.747 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.125551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97977.log 2023-10-18T19:23:28.747 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125551.log.gz 2023-10-18T19:23:28.748 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97637.log 2023-10-18T19:23:28.748 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105284.log.gz 2023-10-18T19:23:28.748 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108388.log 2023-10-18T19:23:28.748 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97977.log.gz 2023-10-18T19:23:28.749 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.97637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92921.log 2023-10-18T19:23:28.749 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.97637.log.gz 2023-10-18T19:23:28.749 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110029.log 2023-10-18T19:23:28.749 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108388.log.gz 2023-10-18T19:23:28.750 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.92921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133001.log 2023-10-18T19:23:28.750 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92921.log.gz 2023-10-18T19:23:28.750 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131625.log 2023-10-18T19:23:28.750 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110029.log.gz 2023-10-18T19:23:28.751 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.133001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109349.log 2023-10-18T19:23:28.751 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133001.log.gz 2023-10-18T19:23:28.751 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.131625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109149.log 2023-10-18T19:23:28.752 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131625.log.gz 2023-10-18T19:23:28.752 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94979.log 2023-10-18T19:23:28.752 INFO:teuthology.orchestra.run.smithi082.stderr: 22.9% -- replaced with /var/log/ceph/ceph-client.admin.109349.log.gz 2023-10-18T19:23:28.752 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.109149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122875.log 2023-10-18T19:23:28.753 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109149.log.gz 2023-10-18T19:23:28.753 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.94979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110549.log 2023-10-18T19:23:28.753 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94979.log.gz 2023-10-18T19:23:28.753 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.122875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98113.log 2023-10-18T19:23:28.754 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122875.log.gz 2023-10-18T19:23:28.754 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129534.log 2023-10-18T19:23:28.754 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110549.log.gz 2023-10-18T19:23:28.754 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115168.log 2023-10-18T19:23:28.755 INFO:teuthology.orchestra.run.smithi082.stderr: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.98113.log.gz 2023-10-18T19:23:28.755 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142946.log 2023-10-18T19:23:28.755 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129534.log.gz 2023-10-18T19:23:28.755 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.115168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129457.log 2023-10-18T19:23:28.756 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115168.log.gz 2023-10-18T19:23:28.756 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.142946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105980.log 2023-10-18T19:23:28.756 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.142946.log.gz 2023-10-18T19:23:28.756 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99015.log 2023-10-18T19:23:28.757 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129457.log.gz 2023-10-18T19:23:28.757 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95022.log 2023-10-18T19:23:28.757 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105980.log.gz 2023-10-18T19:23:28.758 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.99015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104304.log 2023-10-18T19:23:28.758 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99015.log.gz 2023-10-18T19:23:28.758 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118948.log 2023-10-18T19:23:28.758 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95022.log.gz 2023-10-18T19:23:28.759 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130029.log 2023-10-18T19:23:28.759 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104304.log.gz 2023-10-18T19:23:28.759 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118948.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-18T19:23:28.759 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118948.log.gz 2023-10-18T19:23:28.760 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.130029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108866.log 2023-10-18T19:23:28.760 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130029.log.gz 2023-10-18T19:23:28.760 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110936.log 2023-10-18T19:23:28.761 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.108866.log: 22.1% -- replaced with /var/log/ceph/ceph-client.admin.108866.log.gz 2023-10-18T19:23:28.761 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110936.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110936.log.gz 2023-10-18T19:23:28.761 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114872.log 2023-10-18T19:23:28.761 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111281.log 2023-10-18T19:23:28.762 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.114872.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114872.log.gz 2023-10-18T19:23:28.762 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111281.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111281.log.gz 2023-10-18T19:23:28.762 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129284.log 2023-10-18T19:23:28.762 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118307.log 2023-10-18T19:23:28.763 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.129284.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129284.log.gz 2023-10-18T19:23:28.763 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.118307.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118307.log.gz 2023-10-18T19:23:28.763 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112988.log 2023-10-18T19:23:28.763 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135392.log 2023-10-18T19:23:28.764 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.112988.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112988.log.gz 2023-10-18T19:23:28.764 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.135392.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135392.log.gz 2023-10-18T19:23:28.764 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98334.log 2023-10-18T19:23:28.765 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104731.log 2023-10-18T19:23:28.765 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.98334.log: 1.2% -- replaced with /var/log/ceph/ceph-client.admin.98334.log.gz 2023-10-18T19:23:28.765 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.104731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105438.log 2023-10-18T19:23:28.765 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104731.log.gz 2023-10-18T19:23:28.766 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95774.log 2023-10-18T19:23:28.766 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105438.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105438.log.gz 2023-10-18T19:23:28.766 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.95774.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95774.log.gz 2023-10-18T19:23:28.766 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103800.log 2023-10-18T19:23:28.767 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110334.log 2023-10-18T19:23:28.767 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.103800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111068.log 2023-10-18T19:23:28.767 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103800.log.gz 2023-10-18T19:23:28.767 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.110334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111761.log 2023-10-18T19:23:28.768 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110334.log.gz 2023-10-18T19:23:28.768 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111068.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111068.log.gz 2023-10-18T19:23:28.768 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.111761.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111761.log.gz 2023-10-18T19:23:28.769 INFO:teuthology.orchestra.run.smithi082.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-18T19:23:30.150 INFO:teuthology.orchestra.run.smithi082.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-18T19:23:31.278 INFO:teuthology.orchestra.run.smithi082.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-18T19:24:11.606 INFO:teuthology.orchestra.run.smithi082.stderr: 97.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-18T19:24:23.133 INFO:teuthology.orchestra.run.smithi082.stderr: 97.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-18T19:24:23.134 INFO:teuthology.orchestra.run.smithi082.stderr: 2023-10-18T19:24:23.134 INFO:teuthology.orchestra.run.smithi082.stderr:real 0m56.385s 2023-10-18T19:24:23.134 INFO:teuthology.orchestra.run.smithi082.stderr:user 1m44.484s 2023-10-18T19:24:23.135 INFO:teuthology.orchestra.run.smithi082.stderr:sys 0m5.347s 2023-10-18T19:24:23.136 INFO:tasks.ceph:Archiving logs... 2023-10-18T19:24:23.137 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-17_20:38:29-rbd-wip-yuri2-testing-2023-10-11-0853-pacific-distro-default-smithi/7431146/remote/smithi082/log 2023-10-18T19:24:23.139 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-18T19:24:33.988 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_84698e743199450ee5f6969d22ebe6bed3a68e4a/qa/tasks/ceph.py", line 1917, in task osd_scrub_pgs(ctx, config) File "/home/teuthworker/src/github.com_ceph_ceph-c_84698e743199450ee5f6969d22ebe6bed3a68e4a/qa/tasks/ceph.py", line 1312, in osd_scrub_pgs raise RuntimeError('Exiting scrub checking -- not all pgs scrubbed.') RuntimeError: Exiting scrub checking -- not all pgs scrubbed. 2023-10-18T19:24:33.990 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-18T19:24:34.000 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_84698e743199450ee5f6969d22ebe6bed3a68e4a/qa/tasks/ceph.py", line 1917, in task osd_scrub_pgs(ctx, config) File "/home/teuthworker/src/github.com_ceph_ceph-c_84698e743199450ee5f6969d22ebe6bed3a68e4a/qa/tasks/ceph.py", line 1312, in osd_scrub_pgs raise RuntimeError('Exiting scrub checking -- not all pgs scrubbed.') RuntimeError: Exiting scrub checking -- not all pgs scrubbed. 2023-10-18T19:24:34.002 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-18T19:24:34.002 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-18T19:24:34.058 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-18T19:24:34.112 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-18T19:24:34.113 DEBUG:teuthology.orchestra.run.smithi082:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T19:24:34.129 INFO:teuthology.orchestra.run.smithi082.stderr:bash: ntpq: command not found 2023-10-18T19:24:34.132 INFO:teuthology.orchestra.run.smithi082.stdout:210 Number of sources = 4 2023-10-18T19:24:34.133 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-18T19:24:34.133 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2023-10-18T19:24:34.133 INFO:teuthology.orchestra.run.smithi082.stdout:^* hv01.front.sepia.ceph.com 3 6 377 27 +3287us[+3282us] +/- 70ms 2023-10-18T19:24:34.134 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 35 -2295us[-2300us] +/- 71ms 2023-10-18T19:24:34.134 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 30 -962us[ -967us] +/- 63ms 2023-10-18T19:24:34.134 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T19:24:34.135 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-18T19:24:34.144 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-18T19:24:34.145 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-18T19:24:34.154 DEBUG:teuthology.orchestra.run.smithi082:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-18T19:24:34.274 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-18T19:24:34.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T19:24:34.305 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-18T19:24:34.314 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-18T19:24:34.323 INFO:teuthology.task.internal:Duration was 13078.971558 seconds 2023-10-18T19:24:34.324 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-18T19:24:34.333 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-18T19:24:34.333 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-18T19:24:34.388 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-18T19:24:34.592 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-18T19:24:34.593 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi082.front.sepia.ceph.com 2023-10-18T19:24:34.593 DEBUG:teuthology.orchestra.run.smithi082:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-18T19:24:34.652 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-18T19:24:34.653 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-18T19:24:34.732 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-18T19:24:34.733 DEBUG:teuthology.orchestra.run.smithi082:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-18T19:24:34.848 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-18T19:24:34.858 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-18T19:24:34.859 DEBUG:teuthology.orchestra.run.smithi082:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-18T19:24:34.883 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-18T19:24:34.893 DEBUG:teuthology.orchestra.run.smithi082:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-18T19:24:34.946 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = core 2023-10-18T19:24:34.959 DEBUG:teuthology.orchestra.run.smithi082:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-18T19:24:35.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T19:24:35.014 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-18T19:24:35.025 INFO:teuthology.task.internal:Transferring archived files... 2023-10-18T19:24:35.026 DEBUG:teuthology.misc:Transferring archived files from smithi082:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-17_20:38:29-rbd-wip-yuri2-testing-2023-10-11-0853-pacific-distro-default-smithi/7431146/remote/smithi082 2023-10-18T19:24:35.027 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-18T19:24:35.093 INFO:teuthology.task.internal:Removing archive directory... 2023-10-18T19:24:35.094 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-18T19:24:35.138 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-18T19:24:35.150 INFO:teuthology.task.internal:Not uploading archives. 2023-10-18T19:24:35.150 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-18T19:24:35.160 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-18T19:24:35.161 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-18T19:24:35.193 INFO:teuthology.orchestra.run.smithi082.stdout: 262176 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Oct 18 19:24 /home/ubuntu/cephtest 2023-10-18T19:24:35.193 INFO:teuthology.orchestra.run.smithi082.stdout: 661082 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Oct 18 15:59 /home/ubuntu/cephtest/mnt.0 2023-10-18T19:24:35.194 INFO:teuthology.orchestra.run.smithi082.stdout: 661083 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Oct 18 16:00 /home/ubuntu/cephtest/mnt.0/client.0 2023-10-18T19:24:35.194 INFO:teuthology.orchestra.run.smithi082.stdout: 661123 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Oct 18 16:00 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-18T19:24:35.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T19:24:35.195 INFO:teuthology.orchestra.run.smithi082.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-10-18T19:24:35.196 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_84698e743199450ee5f6969d22ebe6bed3a68e4a/qa/tasks/ceph.py", line 1917, in task osd_scrub_pgs(ctx, config) File "/home/teuthworker/src/github.com_ceph_ceph-c_84698e743199450ee5f6969d22ebe6bed3a68e4a/qa/tasks/ceph.py", line 1312, in osd_scrub_pgs raise RuntimeError('Exiting scrub checking -- not all pgs scrubbed.') RuntimeError: Exiting scrub checking -- not all pgs scrubbed. During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi082 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2023-10-18T19:24:35.207 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-18T19:24:35.218 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-18T19:24:35.230 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: RuntimeError: Exiting scrub checking -- not all pgs scrubbed. 2023-10-18T19:24:35.230 INFO:teuthology.nuke:Checking targets against current locks 2023-10-18T19:24:35.262 DEBUG:teuthology.nuke:shortname: smithi082 2023-10-18T19:24:35.262 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-18T19:24:35.287 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-17_20:38:29-rbd-wip-yuri2-testing-2023-10-11-0853-pacific-distro-default-smithi/7431146', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 15:39:35.206702', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOivSv8XGJ5NGv6c6zD4SdEvKnmlMEGAwjHGFVunXQn0rlpezRBKpVUINTonVtsg+ed/sV3IayPLsYHwtPK5Rqw='} 2023-10-18T19:24:35.343 INFO:teuthology.orchestra.console.smithi082:Power off 2023-10-18T19:24:35.344 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-18T19:24:35.367 DEBUG:teuthology.orchestra.console.smithi082:power off output: Chassis Power Control: Down/Off 2023-10-18T19:24:35.368 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T19:24:35.391 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2023-10-18T19:24:39.393 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T19:24:39.503 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2023-10-18T19:24:43.506 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T19:24:43.616 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2023-10-18T19:24:47.622 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T19:24:47.732 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is off 2023-10-18T19:24:47.833 INFO:teuthology.orchestra.console.smithi082:Power off completed 2023-10-18T19:24:47.940 INFO:teuthology.run:Summary data: description: rbd/cli_v1/{base/install clusters/{fixed-1 openstack} features/format-1 msgr-failures/few objectstore/bluestore-stupid pool/small-cache-pool supported-random-distro$/{rhel_8} workloads/rbd_cli_generic} duration: 13078.971557855606 failure_reason: 'Command failed (workunit test rbd/cli_generic.sh) on smithi082 with status 124: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=84698e743199450ee5f6969d22ebe6bed3a68e4a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/cli_generic.sh''' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=a93b79dd5eb549ab8f42c4f9f54257fd status: fail success: false 2023-10-18T19:24:47.941 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-18T19:24:48.022 INFO:teuthology.run:FAIL